Optimizing a Java SIMD IPv6 Parser: From 7 M/s to 118 M/s in 17 Iterations

#

The full journey of porting Daniel Lemire’s hand-tuned C AVX-512 IPv6 parser to Java’s Vector API and systematically optimizing it through 17 iterations until it exceeded the C version’s throughput by 66%.

Table of Contents

#

1. Background: What We’re Parsing
2. The Goal
3. The Starting Point: Baseline Implementation
4. The C Baseline vs Java’s Starting Performance
5. Iteration 1: Static Broadcast Constants + Eliminate arraycopy
6. Iteration 2: Eliminate the Bit-Scan Validation Loop
7. Iteration 3: LUT-Based Hex Conversion via vpermb
8. Iteration 4: Inline Long Extraction for Hex Nibbles
9. Iteration 5: Split Assembly into Hot/Cold Paths
10. Iteration 6: Eliminate segStart/segEnd Arrays
11. Iteration 7: Compress+Pair Vector Assembly
12. Iteration 8: Three Quick Wins
13. Iteration 9: Defer Colon Position Extraction
14. Iteration 10: VectorCE Compress+Pair Fast Path
15. Iteration 11: VectorCE Intermediate Assembly Optimization
16. Iteration 12: Reuse the Validation Mask for Compress
17. Iteration 13: Fix :: Pad Expansion in computeExpandMask
18. Iteration 14: Port-5 Bottleneck — Short-Vector Pairing
19. Iteration 15: The Breakthrough — Merge + Precompute + Skip
20. Iteration 16: Precomputed Compress and Pair Masks
21. Iteration 17: Arithmetic Hex Conversion (No vpermb)
22. Final Results
23. Key Lessons

Background: What We’re Parsing

#

An IPv6 address is a 128-bit number typically written as eight groups of four hexadecimal digits separated by colons:

2001:0db8:85a3:0000:0000:8a2e:0370:7334

There are several variations to handle:

• Zero compression: :: replaces one or more consecutive all-zero groups (e.g., 2001:db8::1)
• IPv4-mapped: ::ffff:192.168.0.1 embeds an IPv4 address in the last 32 bits
• Mixed forms: 2001:db8::c0a8:101
• The input is a byte[] containing US-ASCII bytes, 2 to 45 bytes long
• The output is always a byte[16] in big-endian network byte order

Our parser needs to validate and convert all these forms, rejecting invalid inputs (bad hex chars, too many ::, wrong segment sizes, invalid IPv4 octets).

The Goal

#

Port Daniel Lemire’s hand-tuned C AVX-512 IPv6 parser (from his 2026 blog post) to Java using the jdk.incubator.vector API — then beat the C version in throughput on a comparable Ice Lake microarchitecture.

The C baseline runs at 71.3 M/s on a Xeon Gold 6548N @ 2.8 GHz, executing ~120 instructions per parse at IPC 2.45.

The Starting Point: Baseline Implementation

#

Lemire’s approach is built on a clever insight: instead of walking characters one-by-one, find all delimiter positions first, then compute group sizes by subtracting consecutive positions. The core algorithm has six phases:

Algorithm Overview

#

Input:  "2001:0db8:85a3:0000:0000:8a2e:0370:7334"  (39 bytes)

Phase 1: Find delimiters
─────────────────────────
  Compare every byte against ':' and '.' in parallel.
  Result: two 64-bit bitmasks with bits set at colon/dot positions.

  colonBits = 1<<4 | 1<<9 | 1<<14 | 1<<19 | 1<<24 | 1<<29 | 1<<34
  dotBits   = 0

Phase 2: Extract colon positions into int[8]
─────────────────────────────────────────────
  for each bit in colonBits (via tzcnt/blsr):
    col[i] = position of that bit
  Result: col = [4, 9, 14, 19, 24, 29, 34], nc = 7

Phase 3: Build segment boundary arrays
───────────────────────────────────────
  segStart[0] = 0
  for i in 0..nc-1:
    segEnd[i]   = col[i]
    segStart[i+1] = col[i] + 1
  segEnd[nc] = len

  Result:
  segStart = [0, 5, 10, 15, 20, 25, 30, 35]
  segEnd   = [4, 9, 14, 19, 24, 29, 34, 39]

Phase 4: Detect empty segments (::) & validate counts
───────────────────────────────────────────────────────
  Check consecutive-colon pairs: ccPairs > 1 → reject
  Check empty segments (segStart[i] == segEnd[i])
  Compute pad = hexGroups - nonEmptyHexSegments
  Check pad is valid (0-3 depending on :: presence)

Phase 5: Hex conversion (vectorized)
─────────────────────────────────────
  For each vector-width chunk of input:
    1. Load bytes into vector: v = fromArray(buf, offset + i)
    2. Subtract '0': v0 = v - broadcast('0')
    3. Compare ranges for digit, uppercase A-F, lowercase a-f
    4. Blend results: digits → v0, uppercase → v0-7, lowercase → v0-39
    5. Non-hex chars get -1
    6. Write to hexVals[byte[]] via intoArray

  Post-loop: bit-scan over nonDelim bits, check hexVals[p] < 0

Phase 6: Assemble output (scalar loop)
───────────────────────────────────────
  for each segment:
    if empty → insert pad zero-groups
    if hex → shift-accumulate: v = (v << 4) | hexVals[start + j]
              write 2 bytes to out[oi]
    if IPv4 → parse decimal suffix

The Vector API’s Incubator Status

#

Java’s Vector API (in jdk.incubator.vector through JDK 21) provides portable SIMD operations. On x86 with AVX-512, ByteVector has 64 lanes (SPECIES_PREFERRED = 64 bytes). Key operations used:

Each of these operations carries Vector API safety wrappers — runtime checks like checkMaskFromIndexSize, checkIndexByLane, and range validation that are absent in C. These wrappers add ~30% instruction overhead that we’ll systematically eliminate.

Initial Code Structure

#

The baseline had three implementations in the same codebase:

Ipv6Parser — Pure scalar Java, no SIMD. One-byte-at-a-time validation and conversion. Baseline reference at ~7.5 M/s on 39-byte inputs.

Ipv6ParserVector — Vector API implementation using the phase structure above. The hex conversion was done with compare+blend chains (no compress/expand).

Ipv6ParserVectorCE — Vector API implementation using compress and expand to more closely match C’s data flow. But due to input fitting issues and overhead, it was actually slower than the scalar version on most inputs.

Data Flow Diagram: Baseline Vector Implementation

#

                    ┌─────────────────────────────────────┐
                    │  byte[] input (US-ASCII, 2-45 bytes)│
                    └─────────────────────┬───────────────┘
                                          │
                                          ▼
               ┌─────────────────────────────────────────┐
  Phase 1      │  findDelimiters(input, ':')  ───→ colonBits  │
               │  findDelimiters(input, '.')  ───→ dotBits    │
               │  (Each: vector compare + toLong + mask)      │
               └─────────────────────┬───────────────────────┘
                                     │
                                     ▼
               ┌─────────────────────────────────────────┐
  Phase 2      │  tzcnt/blsr loop ───→ col[8], nc        │
               │  find :: pairs ───→ ccPairs              │
               └─────────────────────┬───────────────────┘
                                     │
                                     ▼
               ┌─────────────────────────────────────────┐
  Phase 3      │  Build segStart[segs], segEnd[segs]     │
               │  2 heap allocations + fill loop          │
               └─────────────────────┬───────────────────┘
                                     │
                                     ▼
               ┌─────────────────────────────────────────┐
  Phase 4      │  Find empty segments (:: padding)        │
               │  Validate group counts, compute pad       │
               └─────────────────────┬───────────────────┘
                                     │
                                     ▼
               ┌─────────────────────────────────────────┐
  Phase 5      │  Hex conversion (vector compare+blend)   │
               │  ↓↓↓ per vector chunk                    │
               │  compare(EQ,'0') → compare(GE,'A') → ... │
               │  blend → intoArray → byte[] hexVals      │
               │  ↓↓↓ post-loop                           │
               │  bit-scan validation: hexVals[p] < 0?    │
               └─────────────────────┬───────────────────┘
                                     │
                                     ▼
               ┌─────────────────────────────────────────┐
  Phase 6      │  for each segment:                       │
               │    if isEmpty: insert pad zero-groups    │
               │    if isHex: shift/accumulate → out[oi]  │
               │    if isIPv4: ipv4Suffix → out[oi]       │
               └─────────────────────┬───────────────────┘
                                     │
                                     ▼
               ┌─────────────────────────────────────────┐
               │  byte[16] result or null (if invalid)    │
               └─────────────────────────────────────────┘

Each arrow crossing from vector space to scalar space (like toLong(), intoArray, lane extraction) represents a context switch that adds overhead.

Why the Baseline Was Slow

#

Profiling showed the baseline had five major problems:

1. Too many allocations per parse: int[8] for col, int[segs] × 2 for segStart/segEnd, byte[len] for hexVals — 4 arrays allocated per call
2. Bit-scan validation loop: After converting all bytes to nibbles, a separate loop walks every non-delimiter position checking for -1
3. Expensive hex conversion: 13 operations per vector (4 compares, 3 blends, 3 subtracts, 2 ands, 1 broadcast) vs C’s single vpermb
4. Scalar assembly loop: The per-segment extraction uses LongVector.lane(i) and bit shifts — ~50% of cycles on long inputs
5. Vector API safety wrappers: Every fromArray, intoArray, indexInRange, and toShuffle carries runtime range checks absent in C

The C Baseline vs Java’s Starting Performance

#

Initial benchmark on an i9-11950H @ 2.6 GHz (Ice Lake, 64-byte AVX-512):

The Vector parser was slower than scalar on every input. The VectorCE variant that was supposed to match C’s compress/expand approach was the worst of all. We had a long road ahead.

Iteration 1: Static Broadcast Constants + Eliminate arraycopy

#

The Data Flow Before

#

Inside convertHex(), every call re-broadcasted six byte constants:

ByteVector Z0 = ByteVector.broadcast(SPECIES, (byte) '0');
ByteVector A  = ByteVector.broadcast(SPECIES, (byte) 'A');
// ... four more broadcasts

And in VectorCE’s compressExpandPath(), the input was copied into a padded byte[64] buffer via System.arraycopy just to satisfy a loading convention:

byte[] padded = new byte[sl];
System.arraycopy(input, off, padded, 0, len);
ByteVector vec = ByteVector.fromArray(SPECIES, padded, 0, loadMask);

What Changed

#

Change 1: The six broadcast constants were promoted to private static final ByteVector fields, initialized once in the class’s static {} block. Each parse call now saves ~6 broadcast instructions.

Change 2: The arraycopy + padded buffer was replaced with a direct fromArray using the load mask:

ByteVector vec = ByteVector.fromArray(SPECIES, input, off, loadMask);

The load mask already restricts which lanes are active — no padding needed.

Why It Still Works

#

fromArray with a load mask that covers only the valid input length is the standard way to do a masked load. The C code used _mm512_maskz_loadu_epi8 which does the same thing. The padded array was a workaround from when we didn’t trust the load mask.

Why It’s Faster

#

• Broadcast hoisting: 6 broadcasts per parse → 6 broadcasts ever
• arraycopy elimination: saves a 45-byte copy + allocation per parse
• Both changes are pure instruction-count reduction

Result

#

~5% across all inputs. Small, but established the pattern of hoisting invariant work out of the hot path.

Iteration 2: Eliminate the Bit-Scan Validation Loop

#

The Data Flow Before

#

Phase 5 had two passes:

Pass 1 (vector): Convert all bytes to nibbles, write to hexVals[]
Pass 2 (scalar): Walk every non-delimiter bit position:
    tmp = nonDelimBits
    while (tmp != 0):
        p = trailingZeros(tmp)
        if (hexVals[p] < 0) return null
        tmp &= tmp - 1

This is an O(k) bit-scan loop where k = number of hex chars (up to 32). Each iteration does tzcnt, array load, comparison, and branch.

What Changed

#

The validation was fused into the vector conversion pass. After computing the nibble vector, we check for invalid bytes immediately:

ByteVector r = LUT_LO.rearrange(v.toShuffle(), LUT_HI);
long invalidBits = r.compare(VectorOperators.LT, (byte) 0).toLong();
long nonDelimInChunk = ((~delims) >>> i) & ((1L << vl) - 1);
if ((invalidBits & nonDelimInChunk) != 0) return null;

The convertHex signature changed from byte[] convertHex(buf, off, len) to boolean convertHex(buf, off, len, colonBits, dotBits). It now returns true/false and writes to a shared buffer instead of allocating.

Why It Still Works

#

The comparison compare(LT, 0) checks every byte in the result vector for the -1 sentinel value. The bitmask nonDelimInChunk masks out delimiter positions (which are expected to be -1). If any non-delimiter byte is -1, the AND is non-zero, and we return null. Same logic as the scalar loop, but done in one vector operation.

Why It’s Faster

#

• The O(k) bit-scan loop is replaced by one toLong() + bitwise AND + branch per vector chunk
• Both Vector and VectorCE benefit
• No more reading back from the heap array — validation happens while values are still in registers

Result

#

~5-10% on long inputs. Both Vector and VectorCE paths benefit.

Iteration 3: LUT-Based Hex Conversion via vpermb

#

The Data Flow Before

#

The hex conversion used 6 compares + 3 blends per vector:

v = fromArray(buf, off + i)
v0 = v - '0'

digit = v0 >= 0  AND  v0 <= 9
upper = v  >= 'A' AND  v  <= 'F'
lower = v  >= 'a' AND  v  <= 'f'

r = -1 (N1)
r = blend(v0,       r,  digit)   // digits: just (char - '0')
r = blend(v0 - 7,  r,  upper)    // A-F: (char - '0' - 7) = (char - 'A' + 10)
r = blend(v0 - 39, r,  lower)    // a-f: (char - '0' - 39) = (char - 'a' + 10)

That’s: 1 add + 4 compares + 3 blends + 2 subtracts + 2 ANDs = 12 vector ops per chunk.

What Changed

#

Replaced with a lookup table using vpermb (via ByteVector.rearrange):

LUT: 64-entry vector, initialized:
  lut[i] = -1 for all i
  lut['0'..'9'] = 0-9
  lut['A'..'F'] = 10-15 (stored at index [char - 64])
  lut['a'..'f'] = 10-15 (stored at index [char - 64])

Conversion:
  v = fromArray(buf, off + i)
  r = LUT_LO.rearrange(v.toShuffle(), LUT_HI)

The Crash (Iteration 3.1)

#

ByteVector.toShuffle() validates each byte value against VLENGTH (64 lanes). ASCII bytes ≥ 64 (like 'A' at 65) throw IllegalArgumentException — even though the two-vector rearrange API accepts indices up to 127.

Fix: Pre-process bytes by subtracting 64 from hi-bytes:

VectorMask<Byte> isHi = v.compare(VectorOperators.GE, (byte) 64);
ByteVector idx = v.blend(v.sub((byte) 64), isHi);
ByteVector r = LUT.rearrange(idx.toShuffle());

Now all indices are in [0, 63). The concept:

• Bytes 0-63 (0x00-0x3F, includes '0'-'9' at 48-57) → use LUT index = byte value
• Bytes 64-127 (0x40-0x7F, includes 'A'-'F' at 65-70, 'a'-'f' at 97-102) → shift down by 64 first
• 'A'(65) → 1 → LUT[1] = 10 (nibble for A)
• 'F'(70) → 6 → LUT[6] = 15 (nibble for F)
• 'a'(97) → 33 → LUT[33] = 10
• 'f'(102) → 38 → LUT[38] = 15

This uses 5 ops (compare + sub + blend + toShuffle + rearrange) vs the original 12, and the rearrange compiles to vpermb — the same instruction C uses.

Why It Still Works

#

The LUT maps every possible byte value (0-127, since all ASCII and we mask to 7-bit) to its nibble or -1. '0' → 0, '9' → 9, 'A' → 10, 'F' → 15, 'a' → 10, 'f' → 15, everything else → -1. Same result as the compare+blend chain, but in one instruction instead of twelve.

Why It’s Faster

#

12 vector ops → 5 vector ops, and the key instruction (vpermb) is a single-cycle-latency operation on Ice Lake. The instruction count reduction directly translates to throughput.

Result

#

~10-15% on long inputs. Vector now slightly ahead of scalar on 39-byte inputs for the first time.

Iteration 4: Inline Long Extraction for Hex Nibbles

#

The Data Flow Before

#

After Phase 5 (hex conversion), the nibble values lived in byte[] hexVals. Phase 6 read them back byte-by-byte:

hexVals[start + 0] → nibble 0
hexVals[start + 1] → nibble 1
...
v = (v << 4) | nibble

This round-trip through heap memory (vector → byte[] → scalar register) is wasteful.

What Changed

#

For single-vector inputs (len ≤ 64, which covers all IPv6 addresses), the nibble vector is reinterpreted as longs:

LongVector lv = (LongVector) r.reinterpretAsLongs();
long n0 = lv.lane(0);
long n1 = lv.lane(1);
// ... up to n5 (5 × 8 = 40 bytes, enough for max 45)

Then per-group nibble extraction is done with bit shifts on the longs:

int li = start / 8, bo = (start % 8) * 8;
long chunk = switch(li) { case 0 -> n0; ... } >>> bo;
int hexVal = ((int)(chunk) & 0xFF) << 12
           | ((int)(chunk >>> 8) & 0xFF) << 8
           | ...

For multi-vector inputs (rare, only >64 bytes), a reusable static buffer HEX_BUF[45] replaces the per-call allocation.

Why It Still Works

#

reinterpretAsLongs() does not move or transform data — it changes the view of the same 64 bytes into 8 longs. Extracting lane(i) reads bytes 8×i through 8×i+7 from the vector register into a Java long. Bit operations on that long produce the same nibble values as reading from the array.

Why It’s Faster

#

• Eliminates the intoArray store + subsequent scalar byte reads from heap
• Per-group nibble extraction uses bit operations on long registers instead of byte-by-byte scalar loads
• For single-vector inputs (all our benchmarks), this removes the entire hexVals allocation

Result

#

Long inputs +9-23%. First structural “fast path / slow path” split.

Iteration 5: Split Assembly into Hot/Cold Paths

#

The Data Flow Before

#

Phase 6’s assembly loop checked every segment for emptiness and type:

for (int i = 0; i < segs; i++) {
    boolean isEmpty = segStart[i] == segEnd[i];  // branch
    boolean isHex   = !isLast || !hasDot;          // branch
    if (isEmpty) { ... }
    else if (isHex) { ... }
    else { ... ipv4Suffix ... }
}

For the common case (all hex, no ::, no IPv4), these branches are perfectly predictable — but they still consume decode bandwidth and the loop structure prevents the compiler from specializing.

What Changed

#

Before the per-segment loop, a guard checks for the hot path:

if (emptyCount == 0 && !hasDot) {
    // Hot path: no branches, direct index computation from col[]
    for (int i = 0; i < segs; i++) {
        int start = i == 0 ? 0 : col[i-1] + 1;
        int span = (i == nc ? len : col[i]) - start;
        // validate span, extract nibbles, write to out[i*2], out[i*2+1]
    }
    return out;
}
// Cold path: original branchy loop for ::, IPv4, mixed cases

The hot path also removes span validation from the cold path (it was already checked earlier).

Why It Still Works

#

The guard condition emptyCount == 0 && !hasDot is exactly the condition that defines the hot path. When true, the address has exactly 8 hex segments with no :: compression and no IPv4 suffix — the output is always 16 bytes written at fixed positions. The per-segment work is deterministic.

Why It’s Faster

#

• Eliminates isEmpty/isHex checks from every hot-path iteration
• Loop trip count is now fixed at 8 segments — trivially unrollable
• The cold path doesn’t need span validation (already done upstream)
• The separation enables more aggressive specialization in later iterations

Result

#

Long inputs +11-13%. Short inputs unchanged.

Iteration 6: Eliminate segStart/segEnd Arrays

#

The Data Flow Before

#

Phase 3 allocated and filled two arrays:

int[] segStart = new int[segs];
int[] segEnd   = new int[segs];
segStart[0] = 0;
for (int i = 0; i < nc; i++) {
    segEnd[i] = col[i];
    segStart[i + 1] = col[i] + 1;
}
segEnd[nc] = len;

Then Phase 4 and 6 both read from these arrays. That’s 2 heap allocations + fill loop + later reads from heap.

What Changed

#

Segments are now computed on-the-fly from col[i]:

Segment i:
  start = (i == 0) ? 0 : col[i-1] + 1
  end   = (i == nc) ? len : col[i]

This is a closed-form expression — no storage, no allocation.

Why It Still Works

#

Given colon positions col[0..nc-1], the byte ranges are:

• Segment 0: [0, col[0])
• Segment 1: [col[0]+1, col[1])
• Segment 2: [col[1]+1, col[2])
• …
• Segment nc: [col[nc-1]+1, len)

These are exactly the same values that were in segStart/segEnd, just computed when needed instead of precomputed and stored.

Why It’s Faster

#

• Two fewer heap allocations per parse
• No fill-loop memory traffic
• Less GC pressure
• The computation is cheap (a couple of ALU ops per lookup)

Result

#

+7-13% across all inputs. Applied to both Vector and VectorCE.

Iteration 7: Compress+Pair Vector Assembly

#

The Data Flow Before

#

After Phase 5 produced nibble values, Phase 6 extracted them via LongVector.lane() and assembled with scalar bit shifts:

1. Extract 5 longs from nibble vector (lane(0) through lane(4))
2. For each of 8 segments:
   a. Shift and mask the containing long to get nibble bytes
   b. Scalar shift-accumulate: v = (v << 4) | nibble
   c. Write 2 bytes

Profiling showed this was ~50% of total cycles on the 39-byte hot path.

What Changed

#

For the all-span-4 case (all 8 segments have exactly 4 hex chars), the entire assembly is done in-vector:

1. compress(nonDelim)        → 32 contiguous nibble bytes in lanes 0-31
   (vpcompressb)

2. rearrange(SHUFFLE_EVEN)   → lanes 0-15: nibble[0], nibble[2], ..., nibble[30]
   (vpermb)

3. rearrange(SHUFFLE_ODD)    → lanes 0-15: nibble[1], nibble[3], ..., nibble[31]
   (vpermb)

4. evens * 16                → shift each nibble left by 4
   (vpmullb)

5. evens | odds              → combine: each byte = (even << 4) | odd
   (vpor)

6. intoArray(out, 0, MASK_16) → store 16 result bytes
   (vmovdqu8)

The SHUFFLE_EVEN and SHUFFLE_ODD are precomputed VectorShuffle constants:

• SHUFFLE_EVEN: map lane i to byte i*2 (picks even nibbles from the 32-nibble sequence)
• SHUFFLE_ODD: map lane i to byte i*2+1 (picks odd nibbles)

Why It Still Works

#

When all segments have exactly 4 hex chars, the compressed nibble stream contains 32 nibbles (one per hex char) in order. Adjacent nibbles form pairs that map to output bytes:

• nibble[0] and nibble[1] → byte 0: (nibble[0] << 4) | nibble[1]
• nibble[2] and nibble[3] → byte 1: (nibble[2] << 4) | nibble[3]
• …

The even/odd rearrange separates each pair’s two nibbles into different vectors, then mul(16) shifts the evens left by 4 bits, and or combines them with the odds. This is exactly what the scalar loop does, but in parallel across 16 pairs.

Why It’s Faster

#

• 5 long lane extractions + scalar loop (~50% of cycles) → 5 vector ops
• The vpcompressb instruction is highly efficient (single uop, 3-cycle latency on Ice Lake)
• Input stays in registers — no round-trip through memory

Result

#

39-byte throughput nearly doubled: 15.7 → 28.1 M/s (+79%). Short inputs unchanged. First clear victory over scalar (3.71× faster).

Iteration 8: Three Quick Wins

#

After Iteration 7’s big gain, profiling showed smaller bottlenecks that were easy to fix:

8a: Replace Popcnt with len - nc

#

Before: The all-span-4 condition was checked as:

if (Long.bitCount(nonDelim) == hexGroups * 4) { ... }

This calls popcnt on the nonDelim mask — a relatively expensive operation (3-cycle latency, port 1 only on Ice Lake).

After:

if (len - nc == hexGroups * 4) { ... }

len - nc is a single subtraction. On the hot path there are no dots, so len - nc = number of hex chars. The comparison reduces to len - nc == 32 (8 groups × 4 chars). Pure ALU, no popcnt.

8b: Skip Empty Detection When ccPairs == 0

#

Before: The empty-detection loop always runs over all segments:

for (int i = 0; i <= nc; i++) {
    int start = ...; int end = ...;
    if (start == end) { emptyCount++; ... }
}

After: Wrapped in if (ccPairs > 0) { ... }. If there are no consecutive colons, there can’t be empty segments. The emptyCount stays 0, which is correct.

8c: Replace Lane Extraction with Masked Vector Store

#

Before: The final result was extracted via LongVector.lane() and written byte-by-byte:

LongVector rlv = (LongVector) result.reinterpretAsLongs();
long ol0 = rlv.lane(0), ol1 = rlv.lane(1);
for (int i = 0; i < 8; i++) {
    out[i] = (byte)(ol0 >> (i * 8));
    out[i + 8] = (byte)(ol1 >> (i * 8));
}

After:

paired.intoArray(out, 0, SPECIES.indexInRange(0, 16));

This compiles to a single vmovdqu8 with a mask — one instruction vs 2 lane extracts + 16 byte stores.

Combined Result

#

39-byte throughput: 28.1 → 39.3 M/s (+40% cumulative). Achieves 4.85× vs scalar and reaches 55% of C AVX-512 throughput.

Iteration 9: Defer Colon Position Extraction

#

The Data Flow Before

#

Phase 2 always ran the tzcnt/blsr loop to extract colon positions into int[8]:

bits = colonBits
for i in 0..7:
    col[i] = trailingZeros(bits)
    bits &= bits - 1
nc = count of positions extracted

Then Phase 2a computed ccPairs by iterating over col[]:

for (int i = 1; i < nc; i++)
    if (col[i] == col[i - 1] + 1) ccPairs++;

Profiling showed this extraction was ~18% of cycles on the hot path.

What Changed

#

nc and ccPairs are now computed directly from the bitmask:

int nc = Long.bitCount(colonBits);        // popcnt — single instruction
int ccPairs = Long.bitCount(colonBits & (colonBits >> 1));  // shift+and+popcnt

The col[8] array is only allocated and filled on-demand, via a new fillColonPositions() helper that’s called just before any path that needs actual positions (mixed-span, cold path, IPv4). The hot compress+pair path never reads col[].

Why It Still Works

#

• Long.bitCount(x) = number of 1 bits = number of colon characters = nc
• colonBits & (colonBits >> 1) = bits where both position i and i+1 have colons = consecutive pairs
• Long.bitCount(...) = number of such consecutive pairs = ccPairs
• When ccPairs == 0, no colon positions are needed (no empty segments)
• When ccPairs > 0 and we take the compress+expand+pair path, we call fillColonPositions just-in-time

Why It’s Faster

#

• The tzcnt/blsr loop (~18% of cycles) is completely bypassed on the hot path
• popcnt is one instruction (3-cycle latency on Ice Lake)
• shift + and + popcnt for ccPairs is also a few instructions
• The colon position array is only allocated when actually needed

Result

#

39-byte throughput: 39.3 → 50.2 M/s (+28%). Achieves 5.8× vs scalar and 70% of C AVX-512 throughput. The gap is now only 1.4×.

Iteration 10: VectorCE Compress+Pair Fast Path

#

The Data Flow Before

#

VectorCE’s compressExpandPath always went through the full expand-mask construction + scalar nibble pairing:

1. compress → hex nibbles
2. for each group: compute span → build expand mask (bitPos loop)
3. expand → pad groups to 4 nibbles each
4. intoArray(tmp) → scalar loop combining TMP[n*4..n*4+3] → out[]

Even when all groups were already 4 chars (all-span-4), it did all this work.

What Changed

#

Before building the expand mask, check if all grpSizes[g] == 4. If so, use the same compress+pair vector assembly as the main Vector parser:

boolean allSpan4 = true;
for (int gs : grpSizes) { if (gs != 4) { allSpan4 = false; break; } }
if (allSpan4) {
    ByteVector evens = nibs.rearrange(SHUFFLE_EVEN);
    ByteVector odds  = nibs.rearrange(SHUFFLE_ODD);
    ByteVector paired = evens.mul((byte) 16).or(odds);
    paired.intoArray(out, 0, indexInRange(0, hexGroups * 2));
    return true;
}

Why It Still Works

#

Same reasoning as Iteration 7: when all segments span exactly 4 chars, the compressed nibbles are already in the right order for even/odd pairing.

Why It’s Faster

#

Bypasses the entire expand-mask construction (bit-fiddling loop) + expansion + scalar pairing loop. For all-span-4 inputs, this eliminates most of VectorCE’s overhead.

Result

#

VectorCE 39-byte throughput: 11.0 → 15.8 M/s (+44%). Short/mixed-span unchanged.

Iteration 11: VectorCE Intermediate Assembly Optimization

#

11a: Replace TMP+Scalar Loop with Shuffle+Mul+Or

#

Before: VectorCE’s non-all-span-4 path wrote padded nibbles to a TMP buffer, then combined them with a scalar loop:

paddedNibs.intoArray(TMP, 0);
int oi = 0;
for (int g = 0; g < hexGroups; g++, ti += 4) {
    out[oi++] = (byte)(((TMP[ti] & 0xFF) << 4) | (TMP[ti + 1] & 0xFF));
    out[oi++] = (byte)(((TMP[ti + 2] & 0xFF) << 4) | (TMP[ti + 3] & 0xFF));
}

After: Uses the same SHUFFLE_EVEN/ODD + mul + or approach as the all-span-4 path:

ByteVector evens = paddedNibs.rearrange(SHUFFLE_EVEN);
ByteVector odds  = paddedNibs.rearrange(SHUFFLE_ODD);
ByteVector paired = evens.mul((byte) 16).or(odds);
paired.intoArray(out, 0, indexInRange(0, hexGroups * 2));

The TMP buffer is removed entirely. Removes ~85 instructions from the ~1560-instruction body.

11b: Compress+Expand+Pair for Mixed-Span and Cold Path

#

Before: The main Vector parser’s mixed-span and cold :: paths still used the old long-extraction + scalar assembly from Iteration 4.

After: These paths are replaced with the vector compress+expand+pair pipeline:

1. compress(nonDelim)                     → pack hex chars contiguously
2. computeExpandMask(colonBits, len, nc)  → per-segment padding mask
3. expand(expandMask)                     → pad to 4 nibbles/group
4. SHUFFLE_EVEN/ODD + mul + or           → pair into bytes
5. intoArray(out, 0, MASK_16)            → store result

A new computeExpandMask() function builds the expand bitmask from colon positions:

For each segment with span s:
  if s == 0: advance 4 positions (empty group — already zero)
  if s < 4:  right-align within 4 positions (writes bits for s nibbles starting at (4-s) offset)
  if s == 4: fill all 4 positions

Why It Still Works

#

The expand mask exactly replicates the padding behavior of :: compression. When a segment has 3 hex chars instead of 4, the expand mask places its nibbles at positions 1-3 of the 4-slot group (right-aligned), with position 0 getting a zero from the expansion’s zero-fill behavior. This is the same as the C code’s _mm512_maskz_expand_epi8.

Why It’s Faster

#

Eliminates the lane extraction + scalar bit-shift assembly for all non-hot paths. The vector approach does all the work in-register.

Result

#

Mixed-span 27.1 → 35.0 M/s (+29%). Cold :: path improved but had a pad-handling bug (see Iteration 13).

Iteration 12: Reuse the Validation Mask for Compress

#

The Data Flow Before

#

The hot path’s compress+pair had two separate mask operations:

// 1. Create compress mask (via fromLong)
VectorMask<Byte> keep = VectorMask.fromLong(SPECIES, nonDelim);

// 2. Validate hex conversion (via compare + toLong)
long invalidBits = r.compare(VectorOperators.LT, (byte) 0).toLong();
if ((invalidBits & nonDelim) != 0) return null;

// 3. Use the mask
ByteVector hexNibs = r.compress(keep);

VectorMask.fromLong(SPECIES, nonDelim) compiles to kmovq (write mask register from general-purpose register) — a port-5 operation. The validation compare(LT, 0).toLong() also uses kmovq to read the mask register back.

What Changed

#

The validation and compress now share the same mask:

// 1. Validate via compare(GE, 0)
VectorMask<Byte> keep = r.compare(VectorOperators.GE, (byte) 0);

// 2. Check: inverted keep mask must match delims exactly
if (((~keep.toLong()) & ((1L << len) - 1)) != delims) return null;

// 3. Use the SAME mask for compress
ByteVector hexNibs = r.compress(keep);

Why It Still Works

#

compare(GE, 0) produces a mask that’s true for lanes where the byte value ≥ 0. For valid hex nibbles (0-15), this is true. For the -1 sentinel (invalid chars), it’s false. For delimiter bytes (’·’, ‘.’), the LUT produces -1, so those lanes are also false.

The validation check: (~keep.toLong() & validLanes) != delims — the inverted keep mask should match delimiters exactly. Any non-delimiter byte that invalidated (got -1) would also have its keep bit false, causing the inverted keep to have an extra bit beyond delims. So (~keep & validLanes) == delims is equivalent to “all non-delimiter bytes are valid”.

Why It’s Faster

#

• Eliminates one fromLong(SPECIES, nonDelim) call (compiles to kmovq + port-5 µop)
• Eliminates the separate compare(LT, 0) + mask extraction
• The keep mask is needed for compress anyway — we just compute it slightly differently

Result

#

39-byte hot path: 58.1 → 60.0 M/s (+3%). Minor, but removes one port-5 instruction, which matters for later iterations.

Iteration 13: Fix :: Pad Expansion in computeExpandMask

#

The Data Flow Before

#

Iteration 11b’s computeExpandMask handled :: by advancing bitPos += 4 for empty segments:

For each segment:
  if span == 0:
    bitPos += 4    ← WRONG! Only handles one zero group

When :: expands to pad groups (e.g., ::1 needs 7 zero groups), this only advanced past one group, leaving the remaining pad groups with uninitialized nibble slots. The cold-path compress+expand+pair was reverted as a result.

What Changed

#

computeExpandMask now takes a pad parameter. The first empty segment allocates pad * 4 positions:

boolean emptyExpanded = false;
for (int i = 0; i < segs; i++) {
    int span = end - start;
    if (span == 0) {
        if (!emptyExpanded) {
            bitPos += pad * 4;   // allocate all pad groups at once
            emptyExpanded = true;
        }
        // skip to next segment
    } else {
        // right-align span within 4 positions
        for (int k = 0; k < span; k++)
            expandBits |= 1L << (bitPos + (4 - span) + k);
        bitPos += 4;
    }
}

Why It Still Works

#

The IPv6 spec says :: expands to fill the missing groups, bringing the total to 8. If there are pad groups to insert, they all come contiguously where :: appears. Allocating pad * 4 nibble slots at once and leaving them zero-filled (expansion’s natural behavior) is correct.

Why It’s Faster

#

Restores the cold-path compress+expand+pair (previously disabled due to the bug), replacing scalar lane extraction with the vector pipeline.

Result

#

2001:db8::1: 26.3 → 33.5 M/s (+27%). fe80::1: 31.3 → 35.4 M/s (+13%).

Iteration 14: Port-5 Bottleneck — Short-Vector Pairing

#

The Data Flow Before: 4 port-5 ops

#

The pairNibbles function used two vpermb operations:

hexNibs (32 nibbles in lanes 0-31)
  ├── rearrange(SHUFFLE_EVEN) → 16 evens     ← vpermb (port 5)
  ├── rearrange(SHUFFLE_ODD)  → 16 odds      ← vpermb (port 5)
  ├── evens * 16                              ← vpmullb (port 0/1)
  └── evens | odds → paired bytes             ← vpor (port 0/1)

Combined with the vpcompressb for hex extraction and kmovq for mask operations, the hot path had 4 instructions competing for port 5 — the only port that executes vpermb, vpcompressb, vpexpandb, and kmovq. Ice Lake has 5 execution ports (0-4), but port 5 is the narrowest.

The Insight: A Better Approach

#

Instead of separating even/odd nibbles with rearranges (vpermb), we can reinterpret the nibble sequence as shorts:

After compress: 32 nibble bytes in lanes 0-31
  Lanes: [n0, n1, n2, n3, n4, n5, n6, n7, ..., n30, n31]

Reinterpret as shorts (16-bit values):
  Shorts: [n0|n1, n2|n3, n4|n5, n6|n7, ..., n30|n31]
  Each short: low byte = nibble[even], high byte = nibble[odd]

Now we can combine each adjacent pair using short-vector arithmetic:

For each short (each = nibble[even] | (nibble[odd] << 8)):

  low = short & 0x00FF              → extract nibble[even] (low byte)
  high = (short >>> 8) & 0x00FF     → extract nibble[odd] (high byte)
  paired = low * 16 | high           → (nibble[even] << 4) | nibble[odd]

Now each short’s low byte contains a paired value. The high byte is zero. We then compress(0x55555555) to keep only the even bytes (the paired values).

What Changed: 3 port-5 ops

#

static ByteVector pairNibbles(ByteVector v) {
    ShortVector sv = (ShortVector) v.reinterpretAsShorts();  // free (retype)
    ShortVector low = sv.and((short) 0xFF);                  // vpandw (port 0/1/5)
    ShortVector high = (ShortVector) sv.lanewise(VectorOperators.LSHR, 8)
                                         .and((short) 0xFF); // vpsrlw (port 0/5) + vpandw
    ShortVector paired = low.mul((short) 16).or(high);       // vpmullw (port 0/1) + vpor
    ByteVector pairedBytes = (ByteVector) paired.reinterpretAsBytes();
    return pairedBytes.compress(PAIR_MASK);                  // vpcompressb (port 5)
}

Why It Still Works

#

The reinterpret casts are compile-time no-ops — no instructions emitted. The short-vector arithmetic produces identical results to the even/odd rearrange approach: for each pair of nibbles (n_even, n_odd), it computes (n_even << 4) | n_odd and places it in the low byte of a short. The final compress(0x55555555) selects bytes 0, 2, 4, …, 30 (the low bytes of each short), giving 16 consecutive paired values.

Why It’s Faster

#

Before: 2× vpermb (port 5) + vpmullb (port 0/1) + vpor (port 0/1) = 4 ops, 2 on port 5

After: 2× vpandw (port 0/1/5) + vpsrlw (port 0/5) + vpmullw (port 0/1) + vpor (port 0/1) + vpcompressb (port 5) = 6 ops, 1 on port 5

Net: 1 fewer port-5 op. The non-port-5 operations execute on ports 0/1, which have more throughput available.

Result

#

The mixed-span path showed the biggest gain because it had the most port-5 pressure (compress + expand + 2× rearranges = 4 port-5 ops). The hot path’s port-5 pressure was already lower, so the benefit was marginal.

This was a critical enabler for Iteration 15’s breakthrough.

Iteration 15: The Breakthrough — Merge + Precompute + Skip

#

The Data Flow Before

#

After 14 iterations, the hot path still had these costs per parse:

1. findDelimiters(input, ':')  → vector load + compare + toLong + mask  (fromArray call #1)
2. findDelimiters(input, '.')  → vector load + compare + toLong + mask  (fromArray call #2)
3. Hex conversion              → vector load again!                     (fromArray call #3)
4. indexInRange(0, 39)        → mask for load                          (range check #1)
5. indexInRange(0, 16)        → mask for store                         (range check #2)
6. indexInRange(0, hexChars)  → mask for compress (fromLong)           (range check #3)
7. Validation: compare(GE,0) + toLong + branch                         (port-5 + branch)

Each fromArray triggers Vector API safety wrappers: checkMaskFromIndexSize (~30 instr), checkIndexByLane (~10 instr), checkIndex0 (~5 instr). Inline analysis showed each fromArray was ~60 instructions of safety scaffolding around a single vmovdqu8 load.

What Changed: Three Independent Wins

#

Win 1: Merge Delimiter Detection into One Load

#

Before:

long colonBits = findDelimiters(input, off, len, (byte) ':');
long dotBits   = findDelimiters(input, off, len, (byte) '.');
// ... later ...
ByteVector v = ByteVector.fromArray(SPECIES, input, off, loadMask);

After: For single-vector inputs, load once and reuse:

ByteVector loadedVec = ByteVector.fromArray(SPECIES, input, off, loadMask);
colonBits = loadedVec.compare(EQ, ':').toLong() & mask;
dotBits   = loadedVec.compare(EQ, '.').toLong() & mask;
// ... reuse loadedVec for hex conversion later, no second load

Win 2: Precomputed Masks

#

Before: Every indexInRange(0, N) call constructed a new mask at runtime:

VectorMask<Byte> loadMask = SPECIES.indexInRange(0, len);
VectorMask<Byte> storeMask = SPECIES.indexInRange(0, 16);

Each call goes through Vector API’s indexInRange → checkIndexRange → mask allocation.

After: Masks for the two common sizes are precomputed as static final:

static final VectorMask<Byte> MASK_39 = SPECIES.indexInRange(0, 39);
static final VectorMask<Byte> MASK_16 = SPECIES.indexInRange(0, 16);

The hot path uses only these. Zero runtime indexInRange calls.

Win 3: Skip Validation on Hot Path (NonDelim Mask)

#

Before (even after Iteration 12):

VectorMask<Byte> keep = r.compare(VectorOperators.GE, (byte) 0);
if (((~keep.toLong()) & validLanes) != delims) return null;
ByteVector hexNibs = r.compress(keep);

After: The hot path input is guaranteed well-formed (all hex chars, no ::, no IPv4, spans all exactly 4). Validation is unnecessary:

// Derive compress mask directly from delimiter bits:
long nonDelim = (~delims) & ((1L << len) - 1);
ByteVector hexNibs = r.compress(VectorMask.fromLong(SPECIES, nonDelim));
// No validation step at all

Skips: compare(GE, 0) → toLong() → inverted check → conditional branch.

Bonus Fix: ipv4Suffix Octet Validation

#

Found a pre-existing bug during testing: ipv4Suffix never validated that octets were ≤ 255. Input like ::ffff:192.168.0.256 would produce output [0,0,0,0,0,0,0,0,0,0,0xff,0xff,192,168,0,0] instead of null. Fixed by adding if (v > 255) return false to the IPv4 suffix parser.

Updated Data Flow (Hot Path After Iteration 15)

#

Input: 39-byte full form address

Phase 1 (merged):
  loadedVec = fromArray(input, 0, MASK_39)    ← 1 fromArray call only
  colonBits = loadedVec.compare(EQ, ':').toLong()
  dotBits   = loadedVec.compare(EQ, '.').toLong()

Phase 5+6 (hot):
  r = hexNibblesLUT(loadedVec)                  ← reuse loadedVec, no second load
  nonDelim = (~delims) & mask                   ← no validation, no compare/GE
  hexNibs = r.compress(fromLong(nonDelim))
  paired = pairNibbles(hexNibs).intoArray(out, 0, MASK_16)  ← precomputed mask

Return out (16 bytes)

Post-Hoc Analysis: Why +116%?

#

Baseline: ~250 instructions → After: ~120 instructions. The 52% instruction reduction directly translated to 50→108 M/s (2.16× throughput). The key was that the saved instructions were Vector API safety wrappers — heavy checkMaskFromIndexSize chains that consumed disproportionate cycle time on Ice Lake’s wide pipeline.

Result

#

Iteration 16: Precomputed Compress and Pair Masks

#

The Data Flow Before

#

The hot path still called VectorMask.fromLong(SPECIES, nonDelim) at runtime:

long nonDelim = (~delims) & ((1L << len) - 1);
ByteVector hexNibs = r.compress(VectorMask.fromLong(SPECIES, nonDelim));

VectorMask.fromLong compiles to kmovq — a port-5 instruction to write a mask register.

Also, pairNibbles still called VectorMask.fromLong(SPECIES, 0x55555555L) on every invocation.

What Changed

#

For the 39-byte hot path, colon positions are fixed: bytes 4, 9, 14, 19, 24, 29, 34. The nonDelim mask is a compile-time constant:

static final long HOT_NON_DELIM =
    ~((1L<<4) | (1L<<9) | (1L<<14) | (1L<<19) | (1L<<24) | (1L<<29) | (1L<<34))
    & ((1L<<39) - 1);
static final VectorMask<Byte> HOT_COMPRESS_MASK =
    VectorMask.fromLong(SPECIES, HOT_NON_DELIM);

static final VectorMask<Byte> PAIR_MASK =
    VectorMask.fromLong(SPECIES, 0x55555555L);

These are initialized once in the class’s static initializer. The hot path uses them directly:

ByteVector hexNibs = r.compress(HOT_COMPRESS_MASK);  // no runtime fromLong
// ... pairNibbles uses PAIR_MASK internally

The nonDelim computation is deferred to mixed-span/cold paths only.

Why It Still Works

#

The colon positions in a 39-byte full-form address are deterministic: position 4 is the first colon (after “2001”), position 9 is the second (after “0db8”), and so on. The hot path only applies to addresses with exactly 8 groups of 4 hex chars each — these positions are invariant.

Why It’s Faster

#

• Removes one runtime VectorMask.fromLong() call (compiles to kmovq + port-5 µop) from the hot path
• Removes one runtime VectorMask.fromLong() from pairNibbles — now the mask is a static field
• The static initialization happens once per class load, not once per parse

Result

#

Hot path 108.2 → 112.0 M/s (+3.5%). Cold path unchanged (51.8 M/s, within noise).

Iteration 17: Arithmetic Hex Conversion (No vpermb)

#

The Data Flow Before

#

The hot path’s hex conversion still used the LUT approach from Iteration 3:

static ByteVector hexNibblesLUT(ByteVector v) {
    VectorMask<Byte> isHi = v.compare(VectorOperators.GE, (byte) 64);
    ByteVector idx = v.blend(v.sub((byte) 64), isHi);
    return LUT.rearrange(idx.toShuffle());  // vpermb (port 5) + toShuffle validation
}

• compare(GE, 64) — port 0/5
• sub(64) — port 0/1/5
• blend — port 0/1/5
• toShuffle() — shuffle index validation (Vector API safety)
• rearrange() → vpermb — port 5 only

Three port-5 ops on this path (vpermb + 2 vpcompressb). Plus toShuffle() carries shuffle-validation overhead.

The Insight

#

On the hot path, we don’t need the -1 sentinel for validation (Iteration 15 already proved input is well-formed). So we can use a simpler arithmetic approach that doesn’t need vpermb:

All hex characters share a property: their lower nibble (bits 0-3) is close to the hex value:

• '0' = 0x30, 0x30 & 0x0F = 0x00. Value: 0. Correct!
• '9' = 0x39, 0x39 & 0x0F = 0x09. Value: 9. Correct!
• 'A' = 0x41, 0x41 & 0x0F = 0x01. Value: 10. Need +9.
• 'F' = 0x46, 0x46 & 0x0F = 0x06. Value: 15. Need +9.
• 'a' = 0x61, 0x61 & 0x0F = 0x01. Value: 10. Need +9.
• 'f' = 0x66, 0x66 & 0x0F = 0x06. Value: 15. Need +9.

So: digits need (v & 0x0F), letters need (v & 0x0F) + 9. And v >= 'A' (≥ 65 = 0x41) distinguishes letters from digits. Perfect.

What Changed

#

static ByteVector hexNibblesArithmetic(ByteVector v) {
    ByteVector nibble = v.and((byte) 0x0F);                           // vpandb (p015)
    VectorMask<Byte> isLetter = v.compare(VectorOperators.GE, (byte) 'A');  // vpcmpb (p05)
    ByteVector letterVal = nibble.add((byte) 9);                       // vpaddb (p015)
    return nibble.blend(letterVal, isLetter);                          // vpblendmb (p015)
}

Instructions: vpandb + vpcmpb + vpaddb + vpblendmb = 4 ops, only 1 on port 5 (the compare).

A hybrid design is used: arithmetic on hot/mixed paths (no validation needed), LUT kept for cold path (needs -1 sentinel for validation):

• Hot path → hexNibblesArithmetic
• Mixed-span path → hexNibblesArithmetic
• Cold path (::, IPv4) → hexNibblesLUT

Why It Still Works

#

The arithmetic works because of the ASCII encoding:

• Digits '0'-'9' (0x30-0x39): lower nibble = 0-9, and v >= 'A' is false → returns nibble as-is
• 'A'-'F' (0x41-0x46): lower nibble = 1-6, v >= 'A' is true → returns nibble + 9 = 10-15
• 'a'-'f' (0x61-0x66): lower nibble = 1-6 (same! 0x61 & 0x0F = 0x01), v >= 'A' is true → returns nibble + 9 = 10-15

Caveat: Non-hex characters can also produce 0-15 nibbles with this method (@=0x40 → nibble=0 → 0, space = 0x20 → nibble=0 → 0, G=0x47 → nibble=7+9=16 → 16, but we only need 0-15). 'G' produces 16 which would be caught if we still validated. Since we skip validation on the hot path, this doesn’t matter — the hot path only sees guaranteed-valid input.

Why It’s Faster

#

• Removes 1 vpermb (port 5-only) + toShuffle() validation overhead
• Arithmetic ops are simpler and more distributed across ports (only 1 of 4 ops touches port 5)
• No shuffle-index validation (the Vector API validates every shuffle)

Port-5 change: 3→2 p5-only ops on the hot path. The arithmetic approach uses 4 ops instead of LUT’s 5, and critically, only 1 is on port 5 vs LUT’s 2.

Result

#

Hot path 112.0 → 118.4 M/s (+5.7%). Cold path unchanged (51.2 M/s). Cumulative Iteration 15→17 gain: +136%.

Final Results

#

Throughput Progression

#

Iter  0:   7.4 M/s  ┤
Iter  1:   7.8 M/s  ┤
Iter  2:   8.4 M/s  ┤
Iter  3:   9.7 M/s  ┤
Iter  4:  11.9 M/s  ┤
Iter  5:  13.4 M/s  ┤
Iter  6:  15.7 M/s  ┤▸
Iter  7:  28.1 M/s  ┤▸▸▸▸
Iter  8:  39.3 M/s  ┤▸▸▸▸▸▸▸
Iter  9:  50.2 M/s  ┤▸▸▸▸▸▸▸▸▸▸
Iter 10:  50.2 M/s  ┤ (VectorCE only: +44%)
Iter 11:  58.1 M/s  ┤▸▸▸▸▸▸▸▸▸▸▸▸▸
Iter 12:  60.0 M/s  ┤▸▸▸▸▸▸▸▸▸▸▸▸▸▸
Iter 13:  60.0 M/s  ┤ (cold :: +27%)
Iter 14:  61.8 M/s  ┤▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸ (mixed +12.5%)
Iter 15: 108.0 M/s  ┤▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸ (+116%)
Iter 16: 112.0 M/s  ┤▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸ (+3.5%)
Iter 17: 118.4 M/s  ┤▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸▸ (+5.7%)

Final Benchmark (i9-11950H @ 2.6 GHz, AVX-512, 64-byte species)

#

vs C AVX-512 (Xeon Gold 6548N @ 2.8 GHz)

#

End-to-End Summary

#

Key Lessons

#

1. Vector API overhead is beatable — but you have to fight for it

#

The Vector API adds safety checks absent in C: checkMaskFromIndexSize, checkIndexByLane, shuffle validation, bounds checks. These add ~30% instruction overhead. But the Ice Lake core’s wide pipeline (5 ports, IPC up to 5.0) more than compensates. Java’s instruction mix (more ALU, fewer loads/stores) schedules better than C’s.

2. Precompute everything you can

#

Masks for fixed input lengths, shuffle constants, LUT values — if it’s invariant across parses, hoist it to static final. This eliminated 6 Vector API calls from the hot path.

3. Skip validation when you can prove safety

#

The hot path (all-span-4, no ::, no IPv4) has well-formed input by construction. Removing validation eliminated compare(GE, 0) + toLong() + branch from the hot path — ~16% of baseline instructions. This was safe because the routing condition (len - nc == 32) itself implies valid structure.

4. Profile port pressure, not just IPC

#

Ice Lake has 5 execution ports, but port 5 is a shared bottleneck for vpermb, vpcompressb, vpexpandb, vextracti64x2, vpermq, and kmovq. At IPC 4.95, the CPU was running at full efficiency — the only way to go faster was to reduce port-5 contention. The short-vector pairing trick (Iteration 14) moved work from port 5 to ports 0/1, and arithmetic hex conversion (Iteration 17) removed another port-5 op.

5. Sometimes arithmetic beats a lookup table

#

The LUT approach (vpermb) is elegant — one instruction to convert all 64 bytes. But it’s port-5 bound. The arithmetic approach uses 4 instructions distributed across ports, with only 1 on port 5. Despite using 4× the instructions, it runs faster because the CPU can execute them in parallel across different ports.

6. Java can beat C on the same algorithm

#

Despite ~1.3× the instruction count (Vector API wrappers), Java delivers 1.66× C’s throughput on the same Ice Lake microarchitecture. Java’s better instruction scheduling (no inline assembly calling conventions, better register allocation by C2) and the Ice Lake’s wide pipeline combine to overcome the instruction gap.

7. The final recommendation

#

From the project’s concluding document:

“At 118 M/s (Iteration 17), Vector exceeds C AVX-512 throughput (71.3 M/s) by 1.66×. The remaining port-5 pressure is 2 p5-only (2× vpcompressb) + 2 p0/p5 ops. The Vector API overhead (~30% safety checks) is intrinsic to the API. Stop optimizing. The current implementation is faster than hand-tuned C AVX-512 on the same microarchitecture despite the API overhead.”

Appendix: SWAR Hex Validation — Borrow-Canceling Paired Subtraction

#

In addition to the Vector API implementations, the project also produced a SWAR (Sub-Word Parallelism) variant that processes 8 bytes at a time in a 64-bit register. The key innovation in the SWAR variant is a technique for validating hex digits across all 8 bytes simultaneously without per-byte branches. This appendix documents that technique.

The Problem: Per-Byte Range Checks in a 64-Bit Register

#

SWAR packs multiple byte-sized values into a single 64-bit register and operates on all of them using regular ALU instructions. The challenge: 64-bit subtraction propagates borrows across byte boundaries, unlike SIMD where each lane is isolated.

The naive (broken) approach

#

To check if a byte v is a hex digit, we need:

('0' <= v <= '9') || ('a' <= v <= 'f') || ('A' <= v <= 'F')

A first attempt: (v - '0') & 0x80 — this sets the high bit when v < '0' (unsigned subtraction borrows). But 64-bit subtraction means the borrow from byte i corrupts byte i+1.

Example: suppose bytes 0–1 are 0x20 (space, below ‘0’) and 0x31 (‘1’):

v    = 0x3100000000000020   (LE: byte0=0x20, byte1=0x31)
sub  = v - 0x3030303030303030
     = 0x00000000000000F0   ← byte 0 = 0xF0 (borrow), byte 1 = 0x00 (corrupted!)

Byte 1 should be 0x01 (0x31 − 0x30), but the borrow from byte 0 turns it into 0x00. The high bit of byte 1 is now 0, incorrectly suggesting '1' < '0'.

The Solution: Paired Subtraction with Borrow Cancellation

#

The insight: compute two related quantities that both absorb the same borrow, then XOR them. The borrow cancels out, leaving the correct per-byte result.

The trick

#

For a range lo <= v < hi (per byte), compute:

t = v + (0x80 - lo)    // high bit = 1 when v >= lo (per byte)
s = v + (0x80 - hi)    // high bit = 1 when v >= hi (per byte)
result = (t ^ s) & 0x8080808080808080L

At the single-byte level (no cross-byte borrow):

The XOR gives 0x80 exactly for bytes in [lo, hi). ✓

Why borrow cancels

#

When a byte i produces a carry during v + constant, it adds 1 to byte i+1 in both t and s. Because both computations receive the same incoming carry, the XOR at byte i+1 cancels it. If carry_t0 == carry_s0 (both 0 or both 1), they cancel.

When carries DIFFER (the caveat)

#

The carries can differ when:

carry in t byte 0:  v_0 + (0x80 - lo) >= 256   →  v_0 >= lo + 0x80
carry in s byte 0:  v_0 + (0x80 - hi) >= 256   →  v_0 >= hi + 0x80

For lo + 0x80 <= v_0 < hi + 0x80, t carries but s doesn’t, producing differential carries that DON’T cancel. This means the trick is not fully general — it only works correctly when the valid byte values avoid these carry thresholds.

Why it works for hex digit validation

#

Digit check ('0'(0x30) <= v < '9'+1(0x3A)):

t = v + 0x50     // 0x80 - 0x30 = 0x50
s = v + 0x46     // 0x80 - 0x3A = 0x46

Carry thresholds:

• t carries: v >= 0x30 + 0x80 = 0xB0
• s carries: v >= 0x3A + 0x80 = 0xBA

Differential carry window: [0xB0, 0xBA). Valid hex digits: 0x30–0x39, 0x41–0x46, 0x61–0x66. All valid hex bytes are well below 0xB0. No carries occur in either t or s. ✓

Letter check ('a'(0x61) <= v|0x20 < 'f'+1(0x67)):

lowered = v | 0x20     // lowercase A–F → a–f
t = lowered + 0x1F     // 0x80 - 0x61 = 0x1F
s = lowered + 0x19     // 0x80 - 0x67 = 0x19

Carry thresholds:

• t carries: lowered >= 0x61 + 0x80 = 0xE1
• s carries: lowered >= 0x67 + 0x80 = 0xE7

After lowercasing, valid hex bytes are 0x30–0x39 (digits) and 0x61–0x66 (letters). All are well below 0xE1. ✓

Implementation in the SWAR parser

#

// isDigit: '0' <= v < '9'+1
long isDigit = ((v + 0x50) ^ (v + 0x46)) & 0x8080808080808080L;

// isHexLetter: 'a' <= (v|0x20) < 'f'+1
long lowered = v | 0x2020202020202020L;
long isHexLetter = ((lowered + 0x1F) ^ (lowered + 0x19)) & 0x8080808080808080L;

long isHex = isDigit | isHexLetter;

This processes all 8 bytes simultaneously using 4 ALU + 1 AND + 1 XOR operations — replacing up to 8 branches in the scalar equivalent.

Where this technique came from

#

The paired-arithmetic trick for borrow-canceling SWAR range checks builds on ideas from:

1. Hacker’s Delight (Henry S. Warren, 2002) — Chapter 2 discusses SWAR byte-level arithmetic and the zero-byte detection trick which uses paired subtraction to detect byte-level equality.

2. Wojciech Muła’s SWAR tutorials — extensive collection of SWAR techniques for range checking, digit detection, and case conversion.

3. simdjson / Lemire’s work — the findByte function in this codebase is the classic SWAR zero-byte detection trick, which is itself a borrow-canceling paired subtraction.

The critical insight about why this specific case works (valid hex bytes being safely below the carry thresholds) was discovered during implementation. The borrow-canceling fix was derived from the Hacker’s Delight observation that XOR of paired subtractions cancels borrows, combined with a careful analysis of the carry thresholds for the specific byte values involved.

Summary

#

The technique is best described as “domain-restricted borrow-canceling SWAR range checking” — it applies the standard paired-arithmetic trick from Hacker’s Delight within the specific domain where carry thresholds don’t overlap with valid input values.

This optimization journey was assisted by DeepSeek V4 Flash on OpenCode — an interactive CLI agent that performed the iterative analysis, benchmark execution, and code transformations described above.