stringzilla

Release: v5.1.1 [skip ci]

Patch

• Fix: Snapshot caller sequences before walking them (1831ccb6)
• Make: Bound the compile workers by available memory (8637f386)
• Make: Link the CUDA objects against the dynamic CRT (d0a002f2)
• Fix: Cache Unicode data atomically for the parallel tests (cdfa4cd4)
• Fix: Guard the device scope in NW and SW cross-products (8584d565)

AES-256 and SHA-256 are some of the world's most common workloads implemented in software and hardware to accelerate encryption and hashing. Both have been optimized for decades and on x86 basically converged to more-or-less the same pure Asm snippets reused in most TLS & cryptography libraries. There are however multiple optimization axis still left to explore:

1. Parallelize across input streams, where you are simultaneously computing checksums of more than one file at a time. To my knowledge, the only other libraries doing that are Intel's ISA-L Crypto and intel-ipsec-mb, and neither is widely used, easy to integrate, or friendly to small inputs. That's one of those areas, where using vanilla Skylake-generation wide AVX-512 intrinsics is more productive than relying on specialized SHA hardware.
2. Better leverage specialized, wider, and newer hardware where possible, like the 4-way AES computation with VAES and faster GHASH for authentication via VPCLMULQDQ. OpenSSL v3.0.13 runs it 1-way, and Rust's Ring v0.17.14 does only 2-way.

Those points come with their challenges and limitations. More on that and performance guidance numbers below.

Hashing: SHA-256

The first axis is the only one SHA-256 has. One digest is a serial chain of 32 dependent SHA256RNDS2 at four cycles each, so no vector width shortens it, and every single-message implementation lands within a few percent of every other — they all ride the same instructions. Sixteen independent messages have no dependency between them at all, and there plain Skylake AVX-512 arithmetic pulls clear of the dedicated silicon:

Hashing, one core	Throughput	Hardware
sha2::Sha256	1.54 GB/s	1 message, SHA-NI
ring::SHA256	1.53 GB/s	1 message, SHA-NI
stringzilla::Sha256, single message	1.41 GB/s	1 message, SHA-NI
stringzilla::Sha256s, dataset order	0.90 GB/s	16 lanes, AVX-512
stringzilla::Sha256s, length-sorted	2.83 GB/s	16 lanes, AVX-512

lanes = sz.Sha256s(16)                         # one hasher per lane
digests = bytearray(len(lanes) * sz.Sha256.digest_length)
lanes.update(batch).digest(out=digests)        # no allocation, GIL released

tags = sz.hmac_sha256(b"secret", messages)     # one tag per message, same kernels

The distance between the last two rows is the one limitation worth knowing. Lanes advance one block per turn, so a group costs as much as its longest member. They retire independently rather than dropping the whole group to scalar when one runs short, but XLSum lines run from 21 bytes to 445 KB, and in dataset order that skew leaves most slots idle. Feeding inputs in length order is the caller's lever, and argsort produces that ordering.

Encryption: AES-256

AES needs no help from the first axis, because counter mode is already parallel inside a single message. Four consecutive counter blocks of one stream encrypt at once, and GHASH follows once the key powers $H^1$ through $H^8$ are precomputed, so the problem reduces to how many blocks an implementation puts in flight per instruction. Profiling makes the ladder obvious, because everyone's hot symbol names its own width: Ring dispatches to aes_gcm_enc_update_vaes_avx2, the OpenSSL that Ubuntu 24.04 ships predates the vaes-avx512 assembly and runs AES-NI one block wide, and we issue _mm512_aesenc_epi128 with _mm512_clmulepi64_epi128.

Encrypting, one core	Throughput	Width
openssl::aes256gcm	2.92 GB/s	1-way, AES-NI
ring::aes256gcm	4.33 GB/s	2-way, VAES on YMM
stringzilla::aes256gcm	6.41 GB/s	4-way, VAES on ZMM
stringzilla::aes256ctr	9.23 GB/s	4-way, VAES on ZMM

Decryption tracks it. Counter mode pulls further ahead than the ladder alone predicts, because OpenSSL hand-wrote AVX-512 for GCM and the chaining modes but never for CTR.

key = sz.Aes256GcmKey(bytes(32))
ciphertext, tag = key.encrypt(b"hello", bytes(12))
assert key.decrypt(ciphertext, bytes(12), tag) == b"hello"

seekable = sz.Aes256CtrKey(bytes(32))          # unauthenticated, and its own inverse
assert seekable.xor(seekable.xor(b"hello", bytes(12)), bytes(12)) == b"hello"

Counter mode takes an absolute byte offset, so a reader after row group four hundred of a million-row file starts there rather than pushing the preceding gigabyte through the AES units. Galois/counter mode adds a tag and gives up seeking to get it, and streaming splits by type rather than by a flag — Aes256GcmEncryptor seals and Aes256GcmDecryptor opens, so a state pointed the wrong way is a compile error rather than a runtime one. Beyond x86 the backends cover Arm crypto extensions and SVE2-AES, RISC-V Zvkned with Zvkg, Power vcipher with vpmsumd, and a WebAssembly path with no cipher instructions at all, which emulates the round function through a constant-time tower-field substitution — the 256-byte lookup table a serial implementation reaches for is itself the side channel.

Both tables come from the StringWars harness over the same XLSum corpus, on one Intel Sapphire Rapids core, against OpenSSL 3.0.13 and Ring 0.17.14. Per-backend and per-size breakdowns live in include/stringzilla/cipher/README.md and include/stringzilla/hash/README.md.

Minor

• Add: Batched HMAC-SHA256 for Rust (dcca5eb5)
• Add: Batched SHA256 and HMAC bindings for Rust and Python (94748339)
• Add: Multi-state SHA-256 for batched hashing (ff12ed3e)
• Add: AES-256 through the C dispatch, Rust and Python (f0abf92c)
• Add: AES-256 counter and Galois/counter modes (c5040538)
• Add: Differential coverage for the byte-sum kernels (64f2dac0)

Patch

• Fix: Keep the Sha256s doctest off third-party imports (67965672)
• Fix: Transpose the SHA256 message window in place (d6aa02e7)
• Improve: Move SHA256 lane state by shuffle rather than gather (84bc9152)
• Improve: Keep every SHA256 lane vectorized to its own end (e8573df5)
• Fix: Keep the whole Ice Lake SHA256 path clear of wide registers (7382cfe0)
• Fix: Reseed tests without the hashing kernels (38309324)
• Fix: Spell GHASH carry-less products for MSVC (725300f9)
• Fix: Keep the Ice Lake SHA256 block clear of ZMM (2eca82c5)
• Improve: Register Python types from one table (d2335f36)
• Fix: Stop punning the SHA256 state through wider types (128e8629)
• Fix: Call platform intrinsics directly for ctz/clz/popcount in ISA kernels (79d9a06e)
• Improve: Spell out the tag compare and the streaming spend path (41584d15)
• Fix: Scrub the streaming state when a Rust encryptor drops (681559d4)
• Improve: Stop widening the scalar hash and cipher tiers (26691dbc)
• Fix: Close the evex512 version window at both ends (c437048e)
• Improve: Reach for the ISA where the cipher fell back to scalar C (5e4a9c34)
• Improve: Write the AES rounds out instead of looping (96ef67ad)
• Fix: Clamp duration histogram bins to the last slot (a1e33d87)
• Fix: Select the RISC-V crypto tier at compile time (b35eb90a)
• Improve: Rebalance and retier the C++ test suite (21693ac6)
• Improve: Rebalance the Rust test suite and de-abbreviate its generics (c40b3de6)
• Fix: Resume UTF-8 delimiter scans at the last emitted match (027c0426)
• Improve: Unify runtime asserts on sz_assert_ (d6231a6b)
• Improve: Build the Rust types in const context (2b4b45c2)
• Fix: Keep the Haswell boundary drain inside its index buffer (cb0f5139)
• Fix: Bound the Haswell LUT reads to their tables (712c6ccf)
• Fix: Write Unicode test caches atomically (a9a63433)

Release: v5.0.7 [skip ci]

Patch

• Improve: Report what the build is doing (09d35568)
• Fix: Serialize the Python object free-lists (a4b00bd3)

Release: v5.0.6 [skip ci]

Patch

• Make: Give nvcc on Windows an MSVC environment (ba0ffa7d)
• Make: Wheel tests race on free-threaded interpreters (#299) (e9accb99)
• Fix: Guard the reverse long-needle search against a loop-bound underflow (9adf85a5)
• Make: Ship a pkg-config file for non-CMake consumers (4f7f1314)
• Make: Anchor the Rust crate and declare its C symbols (83e09597)
• Make: Install StringZilla and StringZillas as CMake packages (67e89bfe)

Release: v5.0.5 [skip ci]

Patch

• Docs: Condense README tables into ASCII blocks (8aeafe49)
• Make: Free-threaded StringZillas wheels and a Windows CUDA build (119dabac)
• Improve: Expose codepoint counting and indexing to JS and Swift (7dca3cec)

• sz::split and the other 13 range helpers took the haystack by value — sz::split(view, "-") never compiled, and an owning lvalue was deep-copied so every returned offset pointed into a private copy and was meaningless against your own iterators. Under SSO the offsets came back small and plausible rather than obviously wrong.
• Rolling fingerprints published uninitialized counts. A chunk shorter than the window completes no hash and never overwrote the min-count accumulators, so results were stack garbage — a 3-byte text reported 1,006,634,040 against 0 from the per-text path — and reused scratch folded one text's counts into the next.
• CUDA contexts are now bound per device and per thread. A current context is per-thread state the driver never creates implicitly, so a scope scheduled on one thread left every other thread with none; kernel tables and timers are keyed by device now.
• CPU capabilities were discarded when no GPU was present. Any driver error other than "no device" returned sz_caps_none_k, so driver-less machines and containers reported no backends at all and silently lost every SIMD tier.
• Similarity scoring fixed on degenerate and wide-cell shapes across every backend, along with gap-track seeding at the boundary and per-pair DP cell sizing.
• Cross-product similarity now batches cells into lockstep launches. Every engine dispatched one cell per scheduler prong while the kernels underneath score 2-64, so a prong filled one lane, idled the rest, and paid the full vector cost — the batched kernels never engaged under the parallel executor. Fixed across serial, NEON, Haswell, Ice Lake and RVV.
• sz_isascii helper is retired in favour of an ISA-specific byteset scan. The old SWAR loop could never reach the SIMD kernels; callers now name the backend explicitly and one that names none fails to compile instead of silently settling for the serial tier. 4.9 ns on NEON vs 7.0 ns for the retired SWAR over 100-byte inputs.
• Packaging: free-threaded Python 3.14 (cp314t) wheels, CUDA-on-Arm wheels building under Clang as host compiler, and the Swift package building on Windows.

Patch

• Docs: Refresh the v5 performance table (065c3c1d)
• Fix: Build CUDA-on-Arm wheels with Clang as the host compiler (8ef6be43)
• Docs: Mention preliminary Java & C# support (4eccd43f)
• Make: Decompose CMake helpers and unify capability naming (9da22cf9)
• Fix: Honor tier overrides, multi-config builds, and stale probe caches (f3378b2d)
• Make: Regroup StringZillas sources by capability tier (a4185d3b)
• Fix: Make StringZillas capability narrowing reversible (9337409d)
• Fix: Keep CPU capabilities when no GPU is present (169ac9a4)
• Fix: Mark cross-product test constants maybe-unused (122cd1ef)
• Fix: Cover degenerate and wide-cell similarity shapes on every backend (7478e427)
• Fix: Guard degenerate cells and match warp kernels to cell width (fa3a4dbf)
• Fix: Align UTF-8 break page LUTs to 64 bytes (d4122ab6)
• Fix: Clear rolling counts before each fingerprint chunk (ff741d4e)
• Fix: Detect host-accessible allocators without concepts (48785e96)
• Fix: Resolve the GPU scope through one binder (fe72a2af)
• Fix: Bind a CUDA context per device, per thread (29db56e3)
• Fix: Local min/max operators for block reductions (f4ffe616)
• Fix: Size per-pair DP cells from the materialized reach (9e10818b)
• Improve: One reach derivation for the candidate-lane tiers (aae78659)
• Fix: Seed discarded gap tracks one step above the boundary (96152db6)
• Improve: Draw fuzzy corpora one UTF-8 character at a time (82d480ee)
• Improve: Skip the rune and affine planes when inputs allow (d0bd1832)
• Improve: Retire sz_isascii for a byteset scan (050d5f67)
• Fix: Inline linkage on sz_rfind_byte_serial (0e21398c)
• Make: Build free-threaded cp314t wheels (3f430acb)
• Improve: Serialize StringZillas engines per object (70eee2e4)
• Improve: Batch cross-product cells into lockstep launches (87294b5f)
• Fix: Borrow range haystacks instead of copying them (be0a7128)
• Fix: Materialize concatenation expressions from views and literals (9dc777ab)
• Fix: Build the Swift package on Windows under clang (cfc5c80c)

Release: v5.0.3 [skip ci]

Patch

• Improve: Close binding coverage gaps in Python & JavaScript (b9d34b05)
• Make: Load libcuda stub during CUDA wheel tests (a904900b)

Release: v5.0.2 [skip ci]

Patch

• Make: Ship npm as per-platform prebuilt packages (982c0b2f)
• Make: Link libcuda from the toolkit stub dir on driver-less hosts (af891e7e)

Release: v5.0.1 [skip ci]

Patch

• Make: Publish the npm package via OIDC trusted publishing (fa8f5ffb)
• Make: Guard the CUDA install against NVIDIA's obsoleting cccl (aa88432d)
• Make: Point the Rust publish at the renamed forkunion submodule (e83a55fb)
• Fix: Detect the C++ standard on MSVC via _MSVC_LANG (657f1bfd)

v4 vectorized sorting, hashing, and edit distances. v5 goes after the part everyone agreed was too irregular for SIMD — Unicode — plus the hardware and packaging that make it portable.

<table> <thead> <tr> <th align="left">Domain</th> <th align="left">StringZilla v4</th> <th align="left">StringZilla v5</th> </tr> </thead> <tbody> <tr> <td><b>Unicode</b></td> <td>SIMD-vectorized case folding, uncased search</td> <td> <b>+</b> NFC/NFD/NFKC/NFKD normalization, <b>+</b> grapheme, <b>+</b> sentence, <b>+</b> word, and <b>+</b> line-break iterators</td> </tr> <tr> <td><b>Bioinformatics</b></td> <td>~700 GCUPS Levenshtein & ~10 GCUPS NW/SW on an H100</td> <td>up to <b>~6 TCUPS</b> Levenshtein and <b>~700 GCUPS</b> NW/SW on the same H100</td> </tr> <tr> <td><b>Portability</b></td> <td>x86 & Arm</td> <td> <b>+</b> WebAssembly SIMD, <b>+</b> RISC-V RVV, <b>+</b> LoongArch LASX, <b>+</b> IBM POWER VSX </td> </tr> <tr> <td><b>Hashing</b></td> <td>on par with xxHash & aHash</td> <td><b>+</b> up to <b>3x faster</b> multi-seed digests of one input — for Cuckoo tables, Count-Min & Bloom sketches</td> </tr> <tr> <td><b>Fingerprints</b></td> <td>streaming ~390 MB/s</td> <td>over <b>700 MB/s</b> on an H100 at the same embedding dimension — for large-scale MinHash content dedup</td> </tr> </tbody> </table>

Major

• Break: Consume ForkUnion v3 through its C API (76a2de74)
• Break: Align empty-needle search with strstr across C, C++, and Rust (7cd19532)
• Break: Harden the Python binding against degenerate and zero-size inputs (6567fd26)
• Break: Redesign UTF-8 segmentation iterators across C++/Rust/Python (afbadfcb)
• Break: Role-based function-qualifier vocabulary (SZ_API_/SZ_HELPER_/SZ_DEVICE_*) (1c581b2d)
• Break: Raise StringZillas minimum C++ standard from C++17 to C++20 (a651d7f1)
• Break: Plural UAX-29 word segmentation API (be4238c7)
• Break: Align kernel & ISA names (a20ffa4b)
• Break: Caller-owned scratch & status-carrying APIs for StringZillas (0915e1cd)
• Break: Switch the byte-level NW/SW C API to 32×32 class costs and add an AVX2 backend (31ca928a)
• Break: Replace dense 256×256 substitution matrix with compact 32×32 class table (498e41ca)
• Break: Split parallel StringZillas into per-ISA and per-algorithm units (fb1f0778)
• Break: Split core into per-ISA headers; add WASM/RVV/LASX/Power & rename crypto/Power backends (4783d633)

Minor

• Add: Wire RVV similarity engines to full NEON parity + faster kernels (5e8d6db3)
• Add: RVV weighted candidate-lane walker for batched NW/SW (3c055713)
• Add: RVV anti-diagonal NW/SW + diagonal Levenshtein scorers (a51bce5f)
• Add: RVV batch lane-per-pair Myers + cross-product Levenshtein engine (92eced37)
• Add: RVV similarities foundation + fix clang rv64gcv scoring-ABI miscompile (9059205a)
• Add: AVX2 uniform-cost Levenshtein scorers for Haswell (c0f0a927)
• Add: Realistic multi-script UTF-8 prose fixtures across the test suites (82f009bc)
• Add: C# / .NET bindings via P/Invoke LibraryImport (e329cefb)
• Add: Java bindings via the Foreign Function & Memory API (8898899a)
• Add: Vectorized sz_find_delimiter_utf8 UTF-8 delimiter scan (b5f70c33)
• Add: Haswell and NEON backends for UTF-8 segmentation kernels (c2560c57)
• Add: Rule-derived UTF-8 test hardening from the UCD tables (ade89e4d)
• Add: GB9c Indic-conjunct support to the grapheme test baseline (0d767c6d)
• Add: Per-family Python test modules with ICU-anchored UTF-8 coverage (3ee31874)
• Add: UAX-29 sentence kernels and noun-based UTF-8 segmentation API (c85cccbd)
• Add: GPU warp-cooperative Myers tier for long/few-pair Levenshtein (67389d04)
• Add: GPU UTF-8 Levenshtein; fix Lev<NW dispatch, affine spills, 100k UTF-8 OOM (47423c89)
• Add: GPU cross-product correctness cases for multi-word Myers reuse (ab165d81)
• Add: UTF-8 unit-cost rune Myers on haswell and neon (6070cdbd)
• Add: i32/u32 candidate-lane kernel bodies for the wider-score tier (e3ed352a)
• Add: Cross-product all-pairs similarity matrices (77887fff)
• Add: Free-lists for Python Str and Strs headers (cb5e7966)
• Add: UTF-8 normalization across language bindings (1c07bbec)
• Add: sz_utf8_norm backends for every remaining ISA + differential fuzz (20a36674)
• Add: AVX-512 sz_utf8_norm backends (Skylake baseline + Ice Lake override) (6767bea7)
• Add: WASM SIMD128 case-insensitive UTF-8 find (511e4a21)
• Add: Reverse-order byteset search on Icelake (814d387b)
• Add: Single-pass UTF-8 normalizer sz_utf8_norm with self-documenting tables (70e5d626)
• Add: AVX-512 lockstep Myers Levenshtein family for Ice Lake (9f7a27eb)
• Add: Multi-language UAX-29 word iteration and byte/Unicode ordering (236ddd36)
• Add: Swift UAX-29 word iteration and byte/Unicode ordering (60f4384f)
• Add: Haswell & RVV string-sort backends (6435cf0d)
• Add: WebAssembly SIMD128 multi-seed hashing & UTF-8 case folding (387c84ef)
• Add: RVV runtime detection & Zvk-accelerated AES/SHA hashing (a0506d6f)
• Add: RVV UTF-8 case-folding & case-insensitive search (077b71f2)
• Add: Multi-seed Hashing (bb229eea)
• Add: NEON sequence arg-sort & pgram-sort via vqtbl4q-compress partition (b207063b)
• Add: NEON u8-cell uniform Levenshtein scorers (Ice Lake parity) (ff374ddc)
• Add: NEON uniform-cost Levenshtein distance backend (linear + affine, byte + UTF-8) (c5289f62)
• Add: NEON rolling Min-Hash fingerprint backend (90dc1dad)
• Add: Case-folding tests for Rust & Python bindings (ec9ac5bd)
• Add: Generative adversarial tests for UTF-8 case kernels (9c9db754)
• Add: Scaffold Haswell UTF-8 case backends (d4541212)

Patch

• Make: Build StringZillas against the local core, publish base first (6335e0b0)
• Docs: Record the Zenodo DOI in the citation metadata (a63e6dfe)
• Improve: Public Rust allocator surface, bump allocator-api2 to 0.4 (0f09a8c8)
• Make: Build and ship Windows ARM64 wheels natively (668a9b92)
• Make: Split emulated wheel builds one arch per runner (be6f2525)
• Make: Build each Pre-Release commit once (2678e73e)
• Improve: Turn the UTF-8 norm constants into enums (b6875b69)
• Fix: Share one UTF-8 fold lead-family enum (7354fff1)
• Improve: Decode multibyte UTF-8 in-register on WebAssembly, drop the serial stride (6b484604)
• Improve: Prove clean UTF-8 windows on POWER VSX without per-lane classify (107312bb)
• Improve: Prove clean UTF-8 windows on LoongArch LASX without per-lane classify (37a2396f)
• Improve: Prove clean UTF-8 windows on x86 Ice Lake without per-lane classify (fae13e5b)
• Fix: Use sz_size_t for the Haswell copy tail counter (769b98d0)
• Fix: Gate the Arm SIMD stack off big-endian targets (2c0e79f5)
• Improve: Prove clean UTF-8 windows on x86 Haswell without per-lane classify (29ec6df7)
• Improve: Decode 3-byte UTF-8 runs with a vlseg3e8 tile on RVV (e82228d3)
• Improve: Skip the class-lookup gather on all-ASCII SVE2 sentence and line-break chunks (64d1a297)
• Docs: Refresh the Arm benchmark column on Graviton 5 (d4e3243a)
• Improve: Tile and prove clean windows in the NEON UTF-8 decoder (fbe1681a)
• Fix: Align Arm dispatch tie-breaks with 128-bit measurements (b07b37fd)
• Make: Record the rejected SVE2 intersect kernel and its numbers (8dc491b2)
• Improve: Load shifted views and gate multi-byte classes in NEON whitespaces (87b3c1e1)
• Improve: Decode mixed-width UTF-8 chunks in a third clean lane (366f1e71)
• Improve: Resolve NEON delimiter bitmaps one distinct high byte at a time (a5177c81)
• Improve: Stream windowed in-register delimiter scans on SVE2 (fc0c5941)
• Fix: Byte-order-correct tail shift in the serial short-hash ladder (e96637a5)
• Make: Package exports & subproject hygiene (99e02fdc)
• Improve: Port UTF-8 case folding and case-insensitive search to SVE2 (a1c9c741)
• Fix: Scan the remaining suffix when a cased-rune probe bails to serial (86f36b74)
• Fix: Pointer-width argsort indices in 32-bit wheel tests (5838265e)
• Improve: Port UAX-29 grapheme clusters to SVE2 and bound the NEON classify (bb46babc)
• Fix: Correct VSX lead-classify table reads, retarget wheel tests (a3ef63c4)
• Improve: Port UAX-14 line-break opportunities to SVE2 (7bd7e2c1)
• Fix: Explicit vector casts for Clang 20 on POWER (211c13be)
• Improve: Port UAX-29 sentence boundaries to SVE2 (5ccc3662)
• Make: Recognize Debian arch names, survive unknown CPUs in builds (6aaaf909)
• Make: Document the local QEMU multi-length and cross-ISA sweeps (3dda6b97)
• Improve: Peek across SVE2 token tiles and un-gate whitespace scans (6a44b751)
• Improve: Stream chunked windows in the SVE2 UTF-8 decoder (500544b5)
• Improve: Add SVE2 byte-set search and an SVE reverse substring search (30db9be3)
• Fix: Walk delimiter bitmaps scalar in the NEON tokenizer (889b0f7c)
• Improve: Retire the hybrid word-break helpers, pin Double_Quote goldens (83535b1c)
• Improve: Port WASM, POWER, and LoongArch word-breaks onto the engine (aeaaac59)
• Improve: Port RVV word-breaks onto the shared windowed engine (b82c5e50)
• Make: Emulated LoongArch, POWER, and WASM coverage in CI (b0d6004a)
• Make: Reject tool attributions in commit messages (708e6a86)
• Improve: Golden vectors for deferred Mid* word-breaks on every backend (bd147665)
• Fix: Apply WB4 and defer WB3c in the RVV word-break fast path (5d01c634)
• Fix: Emit deferred Mid* word-breaks across window edges (ff5ec285)
• Make: Root the SwiftPM C target at the repository (5384023a)
• Improve: Flat gather-fed BMP classifiers for UTF-8 segmentation (0d6584de)
• Fix: Run GPU doctest examples via capability guard instead of skipping (b1a0ff35)
• Improve: Add packed (terminator-free) Arrow view for cuDF columns (692c2c20)
• Make: Native multi-arch CUDA wheels, retire QEMU cross-builds (539326ac)
• Improve: Own CUDA device-wide primitives, drop CUB launchers (639501ff)
• Improve: Restructure and expand the PyTest suite (23edd031)
• Make: Scale down fuzz budgets for the QEMU test sweeps (7d3e5a1a)
• Make: Compile ISA probes from files with keyword arguments (97ca3737)
• Fix: Release the file mapping before deleting in the reinit test (a6d1ee7b)
• Improve: Pick SVE over NEON by measured register width (b4319b09)
• Fix: Name evex512 & popcnt in AVX-512 kernel attributes (124d7e93)
• Make: Exercise the Cargo ISA probes in existing CI jobs (ed1614e3)
• Fix: Include <unistd.h> so Linux-Arm CPU detection actually runs (0b3ed16c)
• Make: Probe ISA compile & run sets in CMake and Cargo (caf2f920)
• Improve: Report full hardware truth from CPU detection (9696ccea)
• Fix: Compile the SVE2 word-break engine as C99 (90893570)
• Improve: Broaden sz_sequence_intersect to tolerate duplicates (74b20a85)
• Fix: Match file output for RISC-V in the cross-build verify step (097feb76)
• Fix: Return an empty matrix for empty Strs cross products (3a16f61b)
• Fix: Size the empty-string similarity cell to avoid 8-bit wraparound (9195047f)
• Fix: Parse capability names by exact match, not prefix (86e24bb1)
• Make: Cross-build and QEMU-run RVV/SVE similarity tests across vector lengths (5621f00d)
• Fix: Declare syscall() on RISC-V under a strict -std via _DEFAULT_SOURCE (4797f069)
• Fix: sz.equal compared its first argument to itself (af81c716)
• Improve: Interleave NEON single-word Myers chains for ILP (0ba5b5f2)
• Improve: Faster NEON UTF-8 segmenter classify and compaction (075c674c)
• Improve: Parametrize and extend the Python binding's degenerate-case tests (6716d210)
• Improve: Native full-width SVE2 UTF-8 word-break and token kernels (fc2dbdda)
• Fix: Use a portable temp path in the write_to doctest (e5784aec)
• Improve: Multi-script UTF-8 examples in docs and tests across bindings (386a432a)
• Improve: Faster rolling-hash fingerprint kernels (f2cf8107)
• Improve: Complete the per-ISA forward declarations in the umbrella headers (1cbb98e6)
• Improve: Wire and cover sz_utf8_uncased_order across all ISAs (d50dbb81)
• Make: Test C# and Java bindings in pre-release CI (93086798)
• Improve: Lazy iterators and zero-copy MemorySegment splits for Java (9a793dc4)
• Improve: Lazy iterators and splitting for the C# binding (21ae19ec)
• Fix: Bind C#/Java FFI to the real sz_utf8_wordbreaks export (46036f38)
• Make: Drive per-ISA tests from file-local backend tables (1321ec80)
• Fix: Exclude U+200B/200C/200D from the UTF-8 whitespace set (f42c471a)
• Fix: Size the test stdout buffer so Windows ucrt accepts it (becdd7a9)
• Improve: Assert the segmenter/decoder contract in the UTF-8 iterators (ca306af6)
• Make: Fetch tags in the versioning job so TinySemVer finds them (040f4b2a)
• Make: Enable Python CUDA builds on Windows/MSVC (8b591538)
• Improve: Restructure the Rust StringZillas build, enable CUDA on Windows (29f7a94c)
• Fix: Gate Rust replace_all helpers behind the std feature (a9b8bd1b)
• Improve: Streaming line-break serial engine, no fixed window (e71882c2)
• Fix: NEON grapheme and sentence classification of malformed UTF-8 leads (c0613853)
• Make: Parallelize MSVC compilation in the Python build (e234f629)
• Fix: NEON local-max fold uses vbslq so GCC accepts the compare mask (c14a8104)
• Fix: NEON word-break declares ill-formed 0xF8-0xFF leads as length 4 (bf03deac)
• Fix: Keep sz_static_assert C99-pedantic-clean and move the window assert to file scope (a2841411)
• Fix: Silence unused hashing fuzz counts on no-SIMD-tier targets (ec6b3422)
• Make: Gate C++23 test targets on real toolchain support (5a91b8dd)
• Fix: Load constant NEON vectors via vld1q_u8 for MSVC ARM64 (81b52544)
• Fix: Upgrade sz_static_assert to use _Static_assert when available (97409972)
• Improve: Narrow line-break byte-start array from sz_size_t to sz_u16_t (05c16360)
• Fix: Shrink serial line-break window from 1024 to 128 codepoints (375a44dc)
• Make: Support MSVC as NVCC host compiler and CCCL 3.x include layout (6eb20b11)
• Improve: Generalize CUB iterator usage for CUDA 13+ / CCCL 3.x (66696bf2)
• Fix: Replace GCC-only [[gnu::noinline]] with portable SZ_NOINLINE macro (091f42bf)
• Fix: Explicit instantiation formatting and missing noexcept specifiers (79f89f94)
• Fix: NEON sz_move corruption on overlapping forward moves (374d8e3d)
• Fix: NEON out-of-bounds LUT read in ASCII Word_Break classifier (47843e6e)
• Fix: CI -Werror/portability nits (a_rune init, hash_serial sz_unused_, execinfo musl guard) (e4666ec9)
• Make: Run pre-release CI on all main* branches (c890d21a)
• Improve: Expose the utf8_delimiters tokenizer to Rust, C++, and Python (49b08aaf)
• Improve: Expose UTF-8 rune counting and normalization to Go (744aa73a)
• Fix: Restore C++11/14 compilation of the range views (2eb7c8cc)
• Docs: Restructure READMEs; add similarities & fingerprints module docs (e232cae2)
• Improve: Rename sz_utf8_valid -> sz_utf8_find_malformed (83748d6c)
• Improve: Fuse delimiters into tokens as a segmenter (d0a58754)
• Make: Rust dynamic-dispatch feature with compile-time SIMD dispatch (e856f1d5)
• Fix: Wire Haswell and NEON NW/SW kernels into the similarity benchmark (e9e0fc90)
• Fix: Guard public engine returns with SZ_NOIPA against GCC -O3 miscompile (e1c70877)
• Fix: Sanitize benchmark failure-log filenames (9f9f7796)
• Make: Link libm for CUDA benchmark targets (6016de9b)
• Fix: Normalize the order benchmark baseline and count only scanned bytes in find_nth (ce7c352b)
• Improve: Single-fixup barrett reduction in the floating rolling hash (5f6e4120)
• Fix: Drop rolling-hash debug asserts from Release builds (3d014dfd)
• Docs: Initial tables for operation families (e8d7efd7)
• Docs: Add CITATION.cff for software citation metadata (f7943509)
• Improve: Mutable metadata_ref accessor and span-based try_argsort coverage (04e53db4)
• Improve: Wire utf8_norm and utf8_delimiters into tests and benchmarks (90274c2f)
• Improve: Instrument benchmarks to report hashes/s and comparisons/s (b85cc7e6)
• Improve: Cover all implemented ISAs in the UTF-8 iteration benchmarks and tests (0a3092f8)
• Fix: Align Haswell grapheme and sentence kernels with serial on ill-formed UTF-8 (20f10c28)
• Fix: Compare the full candidate length in the order benchmark (9bf71867)
• Docs: Per-binding READMEs and v5 main-README refresh (c27d9528)
• Improve: Restore conditional wide-byte gathers in the NEON rune drain (flat, perf-neutral) (813af50d)
• Improve: Restore conditional wide-byte gathers in the LASX rune drain (flat, perf-neutral) (524d431b)
• Improve: Restore conditional wide-byte gathers in the Ice Lake rune drain (flat, perf-neutral) (06b9e209)
• Improve: Restore conditional wide-byte gathers in the AVX2 rune drain (flat, perf-neutral) (83032a98)
• Improve: Sort partition/export and NEON/PowerVSX memory kernels (df75303b)
• Improve: Reverse byteset search, RVV ordering, and find/compare laggards (f44766e9)
• Improve: In-register hash buffer merges and batched SVE bytesum (a34f58ba)
• Fix: SVE/x86 kernel correctness and Haswell bytesum high-byte undercount (ac0ad81c)
• Improve: Single signed compare for UTF-8 codepoint counting (77bc1451)
• Improve: Inline serial tail for short Ice Lake table lookups (697ad500)
• Improve: In-register prefix assembly in Ice Lake sequence intersect (543f96c4)
• Improve: Register-only streaming hash merge on Ice Lake (5593fe50)
• Improve: Start-compact the words AVX2 and NEON classifiers (413c7acb)
• Improve: Start-compact the words Ice Lake classifier cold path (0758d584)
• Make: Parallel, header-aware Python extension builds (f57bd293)
• Improve: Faster CJK & 2-byte UTF-8 segmentation on Ice Lake (477d48f2)
• Improve: Streamline UTF-8 segmentation test suite (1fd79e8a)
• Improve: Harden UTF-8 segmentation test suite (6c60999f)
• Improve: O(1)/codepoint serial sentence & linewrap backends (46fe8ded)
• Docs: Passing capability tuples to CPython (9de2498b)
• Improve: Linewraps carry bit-pack + LB21a/LB10 fixes; sentence dense decode (08e8e157)
• Improve: O(n) forward-pass word serial + fix two Ice Lake word window-edge bugs (05fd48aa)
• Improve: Vectorized UAX-14 line-break Ice Lake kernel (60dc54ec)
• Fix: Ice Lake SB8 boundary lost when an ATerm neutral right-context outruns one window (85ae8fee)
• Improve: O(n) forward-pass serial sentence segmenter (4x), matching the word/grapheme shape (d33ee83e)
• Improve: Gate the sentence page LUT and trie behind lane-presence (20ec4492)
• Improve: 4-bit-pack word astral_leaf via the shared nibble cascade (865e6f2e)
• Improve: 4-bit-pack grapheme stage_sub to halve the residual cascade (cee39091)
• Improve: CJK/Kana arithmetic fast-path skips the grapheme cold cascade (7e5312c6)
• Improve: Default UTF-8 iterator batch STEPS 16 -> 64 (c4c97bb1)
• Improve: Re-init-free Ice Lake grapheme classifier, O(n) serial driver, branchless carries (eddb02fa)
• Improve: Re-init-free Ice Lake word classifier reads tables from aligned .rodata (3083e6fc)
• Improve: Register-resident astral trie for the Ice Lake word kernel (982381f8)
• Make: Normalize UTF-8 boundary APIs onto sz_utf8_find_linewraps (1c49c9fa)
• Improve: GPU Myers reuse cap to 2048 recovers the 1KB Levenshtein cliff; parallel wheel build (5fd964c5)
• Improve: Route 129-512 byte weighted NW/SW pairs to the warp kernel (092f13e1)
• Improve: Multi-word Myers Peq-in-shared reuse kernel for 65-256 byte queries (cf0c7f0b)
• Improve: Size similarity bench batch from STRINGWARS_BATCH_PER_CORE (b535b934)
• Improve: Spread Myers Peq-reuse cross-product across warps for few queries (ba15528d)
• Fix: Reserve a load-transaction word for GPU diagonal scratch (42cc5a7f)
• Improve: INTER-batched UTF-8 rune kernels for non-unit & affine Levenshtein (ab5fda71)
• Fix: Return cuda_status_t via memory to dodge NVCC struct-return bug (1d9651de)
• Make: Candidate-lane cross-product driver generic over element type (659cfe3a)
• Improve: GPU router groups tasks into dyadic length bands (331b2a9b)
• Improve: Hopper DPX in the register/batch weighted GPU kernels (d519ae78)
• Improve: width-tiered candidate-lane driver activates the i32/u32 tier (ead1302f)
• Improve: Zero-copy out= buffer for Strs.argsort (b6e1a4bc)
• Fix: Align packed hash state through an ISA-native aligned twin (29bcaafe)
• Fix: UBSan-clean unaligned access in Haswell & LASX memory kernels (2bda78bf)
• Improve: Flatten and nativize the UTF-8 iterator kernels (b53b3b49)
• Fix: Streaming hash dropped the trailing partial block (a0a87871)
• Fix: Define lossless malformed-UTF-8 handling; consolidate the rune codec (6e43972e)
• Improve: Single-pass UTF-8 newline/whitespace/word iterators across all ISAs (56777b89)
• Improve: Fuse Ice Lake Myers recurrence boolean ops into VPTERNLOGQ (bb2b2432)
• Improve: GPU large-input similarity — tiled device tier + Myers (a3c0382b)
• Improve: Self-contained, fully-masked RVV kernels & cleanup (c38cfbc5)
• Make: Align C/C++/CUDA builds at -O2; skip C rebuild on Rust-only edits (18a3acda)
• Improve: Pin RVV fold mask policy to mask-undisturbed (b49394b4)
• Fix: Guard intersect & lookup on 32-bit / WebAssembly targets (966e7bd1)
• Make: Expose the WebAssembly target to the Rust build & CI (cadf1c57)
• Improve: Cap-free RVV search & streaming word boundaries (dc34751c)
• Improve: Vectorize & clarify UAX-29 word kernels (985c4a7c)
• Improve: Case-insensitive arg-sort on the SIMD partition across all tiers (b52e0987)
• Improve: Introduce Python DocTests, including in CI (d30050f0)
• Docs: Expand & normalize Python docstrings (3e71dbe3)
• Improve: Declare all CPU similarity backend aliases uniformly (834a66c4)
• Make: Stable arg-sort with reverse, top-K & case-insensitive UTF-8 order (862afee7)
• Improve: Benchmark Levenshtein under both unit and non-unit cost schemes (34038e1d)
• Fix: Order bytes as unsigned across serial & Skylake comparisons (10ada8b2)
• Improve: Use vmovl_high for NEON similarity-scorer widening (b4bbc4a3)
• Improve: TBL delta-LUTs for NEON central-European & Greek fold callbacks (2cc1e682)
• Improve: Vector-domain NEON case-insensitive alarms (drop per-value movemasks) (f85a7843)
• Improve: Symmetric Armenian, Georgian & Greek UTF-8 case folding across SIMD tiers (4f1fc166)
• Fix: Enforce cumulative SIMD tiers so a tier implies its substrate (2aff3483)
• Improve: NEON case-insensitive UTF-8 search (12e97b81)
• Fix: Expose 128/256-bit vector union members under all x86 SIMD tiers (46243e88)
• Improve: NEON Cyrillic, Greek & guarded UTF-8 case folding (6defd082)
• Improve: AVX2 case-insensitive search for remaining scripts (0d059960)
• Improve: AVX2 case-insensitive UTF-8 search (cefeee2d)
• Improve: NEON Latin UTF-8 case folding (d8e1a10f)
• Improve: AVX2 UTF-8 case folding for Cyrillic, Greek & CJK (0b60ec58)
• Improve: AVX2 UTF-8 case folding for ASCII & Latin (66bfc903)
• Improve: NEON UTF-8 case folding for caseless scripts (ac00e181)
• Improve: Walker-owned layout() scratch & <=512 Myers (31a92efc)
• Improve: Tune Ice Lake case-search microkernels (3b20e03d)
• Fix: Case-fold crash & Unicode 17 classifier gaps (6bb7a762)
• Improve: Reorganized UTF-8 case kernels for Ice Lake (1ee6379d)
• Fix: Off-by-one in arrow_strings_tape offsets growth (b6721b48)
• Fix: Live timing in benchmark repeat loops (1c5025a7)
• Make: Tidy StringZillas CMake library graph (a64c4418)
• Make: Precompiled StringZillas static & shared libraries (fb8c9d27)
• Improve: Resolve grids via CUDA driver API, not runtime (4acb3c5d)
• Improve: Linear memory serial baselines (960db38a)
• Improve: Deduplicate engine constructors (6be92db6)
• Improve: Own GPU timing events per engine and move launch sync to the driver API (7e9c0984)
• Improve: Affine-gap class-cost NW & Smith-Waterman scorers for Haswell and NEON (7be4867c)
• Fix: Transpose class-cost lookups when the serial walker swaps strings (33d79817)
• Improve: Haswell AVX2 anti-diagonal class-cost NW & Smith-Waterman scorers (5d98cb45)
• Improve: Affine-gap register aligners for short CUDA Levenshtein/NW/SW (ea8b03d2)
• Improve: ARM NEON anti-diagonal class-cost NW & Smith-Waterman scorers (abc182bc)
• Improve: Affine-gap Ice Lake NW/SW class-cost diagonal scorers (de72dca5)
• Improve: Narrow per-string lengths and warp-loop counters to 32-bit in the CUDA aligners (d672b371)
• Fix: Prefer the Hopper DPX tier over base CUDA in the similarity dispatch (2557d3f8)
• Docs: Document fast test/benchmark iteration knobs (6a31e446)
• Make: Scale the UTF-8 case-fold fuzz with SZ_TESTS_MULTIPLIER (298abf21)
• Make: Shrink fingerprint equivalence inputs and generate them once (7c978c48)
• Improve: Run the core test suite through the shared timed runner (78891c94)
• Improve: Shared test runner and coverage for the new backends (c938f3e0)
• Improve: Cap loaded tokens and override batch sizes from benchmark env vars (367da1a7)
• Improve: Exchange fingerprint chunks via warp shuffles and reuse CUDA scratch (8de0a334)
• Improve: Register-resident short-sequence GPU aligners and leaner launches (34bb6350)
• Improve: Fuse the Hopper min-plus recurrence into a single DPX instruction (bd628c1b)
• Improve: Fuse the floating rolling-hash discard and intake into one Barrett reduction (4ce1a597)
• Improve: Ice Lake diagonal SIMD scorers for class-cost NW & Smith-Waterman (266bfc3e)
• Improve: Shared test runner with timing, name filtering, and crash backtraces (6c1529b8)
• Improve: Per-dimension hasher seeding through constructors (5997d8ac)
• Improve: Seed fingerprint hashers from independent splitmix64 streams (d86d57d5)
• Improve: De-scalarize RVV/NEON hash loads and vectorize RVV find_nth (4f63a123)
• Improve: WASM v128/v128relaxed kernels — vectorize hash/SHA-256, shed serial loops (9ba92551)
• Improve: LASX SIMD kernels and add UTF-8 iteration benchmarks (7ebd3b64)
• Make: Refresh Python, Rust, Node, and Swift builds for the per-TU C layout (d3d5335c)
• Make: Cross-compile toolchains, build wiring, hook, and formatting (5a8da703)

Release: v4.6.3 [skip ci]

Patch

• Fix: Guard intersect & lookup on 32-bit / WebAssembly targets (c88afd4e)
• Fix: Parse capability names by exact match, not prefix (7082fdb4)
• Fix: Off-by-one in arrow_strings_tape offsets growth (b1c121d6)
• Fix: sz.equal compared its first argument to itself (f2c3d351)

• Improve: Support free-threaded Python 3.14t (#298) (03820e6)
• Make: Refresh JS & Swift compatibility (04db605)
• Improve: Report failure codes in benchmarks (40842b7)
• Make: Support for Python 3.14 (5606001)
• Improve: Generalize bench environment logging (8144682)
• Fix: Reset szs.capabilities under QEMU (59a0556)
• Make: Don't expect package-lock.json in CI (20f3d26)
• Fix: Silence SwiftPM unhandled-file warning in StringZillaTests (5655376)
• Make: Bump third-party GitHub Actions to current latest (9dd039c)
• Fix: Drop -lcuda from stringzillas-cuda wheel link line (534d3f3)

Release: v4.6.1 [skip ci]

Patch

• Make: Gitignore .claude/ per-project settings (37c2d8f)
• Fix: Free table_positions in sz_sequence_intersect_serial and _ice (087ad7f)
• Fix: OOB read in sz_order_skylake tail for length-mismatched strings (a3926e2)
• Fix: Signed char handling in sz_find_byte_serial (#308) (267c757)
• Fix: OOB in sz_find_skylake / sz_rfind_skylake tail for null-byte needles (#312) (7d72c96)

• New fast case-insensitive substring search path for Georgian 🇬🇪
• Faster hashing with compile-time dispatch w/out stack protection 🔞
• Faster byteset search on Haswell CPUs, thanks to @Caturra000 👏
• Fewer inline-related warnings in C++ builds, thanks to @inflooper 👏

Minor

• Add: Georgian fast path (e02cb00)

Patch

• Fix: inline static warnings with C++23 modules (#287) (374adbf)
• Improve: Reduce startup overhead for sz_find_byteset_haswell (#293) (7f2899a)
• Fix: Missing Georgian dispatch (5f91e7e)
• Improve: Drop stack-protection in hashing on GCC (23801e4)
• Improve: Reduce repeated reviews (2e5784b)
• Improve: Faster sz_size_bit_ceil (4003057)
• Improve: Avoid ZMM-to-stack spill in Skylake comparisons (d94a010)
• Make: Use relative install path for C sources (690d775)

Patch

• Fix: Shared library function signatures (4b6f40f)
• Make: Ignore blaming for recent chores :) (8a4cacd)
• Fix: Cleaner haystack/needle buffer (de44c05)
• Improve: Python iterator over case-insensitive matches (3782f9a)
• Make: NPM Trusted Publishing in CI (738b07f)

<img width="4650" height="2325" alt="search-utf8-example" src="https://github.com/user-attachments/assets/f4886da8-3590-4cc2-a666-368222dc0331" />

Below are the performance numbers comparing the search throughput of unique "word" tokens across various languages of the Leipzig Wikipedia Corpora for a case-insensitive substring search that respects all Unicode 17.0 case-folding rules. This is arguably the only library providing full Unicode spec compliance for search operations besides the PCRE2 library, which is often order(s) of magnitude slower than even our serial baseline due to the extreme complexity of combining a complete RegEx engine with Unicode compliance.

Corpora Language	Script	Serial Baseline, GB/s	AVX-512 for Ice Lake+, GB/s	Speedup
Latin (Basic)
🇬🇧 English	Latin	1.15	10.93	11.9×
🇮🇹 Italian	Latin	0.81	10.63	14.7×
🇳🇱 Dutch	Latin	0.85	10.91	13.3×
Latin (Extended)
🇩🇪 German	Latin+ß	0.74	9.36	13.6×
🇫🇷 French	Latin+Acc	0.73	8.37	15.1×
🇪🇸 Spanish	Latin+ñ	0.99	8.86	10.8×
🇵🇹 Portuguese	Latin+Acc	0.77	9.58	14.3×
🇵🇱 Polish	Latin+Ext	0.62	7.51	14.2×
🇨🇿 Czech	Latin+Háčky	0.43	6.10	17.1×
🇹🇷 Turkish	Latin+İ/ı	0.81	6.78	11.7×
🇻🇳 Vietnamese	Latin+Tones	0.41	6.38	17.9×
Cyrillic
🇷🇺 Russian	Cyrillic	0.54	3.41	10.6×
🇺🇦 Ukrainian	Cyrillic	0.56	4.03	10.6×
Greek
🇬🇷 Greek	Greek	0.31	7.04	22.5×
Caucasian
🇦🇲 Armenian	Armenian	0.34	4.18	17.5×
🇬🇪 Georgian	Georgian	0.65	10.56	24.2×
Semitic
🇮🇱 Hebrew	Hebrew	0.65	9.52	13.7×
🇸🇦 Arabic	Arabic	1.17	9.85	9.8×
🇮🇷 Persian	Arabic+Ext	0.41	11.83	43.1×
Indic
🇮🇳 Hindi	Devanagari	1.25	10.99	16.3×
🇧🇩 Bengali	Bengali	0.72	11.03	25.9×
🇮🇳 Tamil	Tamil	1.09	11.70	21.0×
CJK & East Asian
🇯🇵 Japanese	CJK+Kana	0.52	11.56	26.7×
🇰🇷 Korean	Hangul	2.98	11.58	3.5×
🇨🇳 Chinese	CJK	0.43	20.07	103.0×

Minor

• Add: Case-folding for Swift (f621419)
• Add: Case-folding for GoLang (2e310d6)
• Add: Case-folding for JavaScript (93bc9c4)
• Add: Reusable case-insensitive needles with metadata for Rust & C++ (45e3c92)
• Add: Reusable case-insensitive finder for Rust (0c8626d)
• Add: Vietnamese fast path (8ec7de7)
• Add: Armenian & Greek paths (984ace2)
• Add: Central European block (f087711)
• Add: New Cyrillic search kernels (b8e106f)
• Add: Serial verification for Ice Lake search (0cfd2fd)
• Add: Seeding & iterations multipliers for fuzzing (d96cee7)
• Add: sz_utf8_case_agnostic API (a0507ee)
• Add: Fast-path Ice Lake case-insensitive substring search for needles <= 16 bytes (b836970)
• Add: ASCII fast paths for small inputs (8b136f3)
• Add: Hash-free search kernel for small needles (e5c477d)
• Add: Latin-1 case-folded search (c1c0305)
• Add: Branchless .empty() for small strings (ea258c1)
• Add: Draft TR29 Unicode word-bound iterators (3ca6695)
• Add: Draft case-insensitive search on Ice Lake (4d30daa)

Patch

• Fix: Pointer cast in GoLang (fb20b6c)
• Docs: Badges, CLI, & inconsistencies (b9aa985)
• Docs: UTF-8 Fold & Search with PErf numbers (b69c49d)
• Improve: Prefetch on massive inputs (48a8ccb)
• Fix: Missing span::operator==0 for new NVCC benchamrks (9b6911a)
• Fix: Shaddowing template param on NVCC (908d7f9)
• Fix: Rust UTF-8 iterator doctest (7fff78b)
• Make: Install curl on Alpine for Rust kit (85af5b5)
• Docs: Arm NEON case-folding plans (8a3f25d)
• Improve: Generalize case-invariant logic (1239bea)
• Improve: Faster ASCII kernels for ≤ 3 probes (e6626a8)
• Improve: Deduplicate benchmark input tokens (68287fc)
• Improve: Western European register pressure (fe94e1c)
• Improve: Greek alarm with less register pressure (22dc88c)
• Improve: Separate "alarm" functions for danger zones (936fc22)
• Make: Install bash on Alpine for Rust toolchain (999ec64)
• Improve: Flatten danger zone checks (a8e3f66)
• Improve: VPSHUFB & VPTERNLOG for search (a315ee8)
• Make: Bump Rust & Go CI (7847671)
• Improve: Higher-efficiency Ice Lake kernels (dce1773)
• Fix: Generalize static asserts to 32-bit archs (bde1fad)
• Fix: NULL missing - use SZ_NULL (7e3dd35)
• Fix: Handle failed downloads of UCD specs (13bc864)
• Fix: Micro case-fold in Georgian path (09ca314)
• Fix: Missing sized_match_t constructor (916b23e)
• Improve: Test more problematic chars (46b7135)
• Fix: Shrink step proportional to danger zones (e67f3f7)
• Fix: Outdated case-insensitive metadata in Rust (45e81f7)
• Fix: Missing danger marker in Western kernel (cbcd685)
• Fix: Mid-rune serial matches (be360c7)
• Fix: Vietnamese old-even fold in ZMM (409a44d)
• Fix: Modifiers exclusion from case-less chars (5948374)
• Fix: Danger zone length (be52b86)
• Improve: Reproduce Ice Lake bugs (ac31704)
• Fix: Eastern European case folding (900033e)
• Fix: Dispatch Central European path (2a8a9b6)
• Fix: Ban "ss" prefix/suffix for Western European path (66cade8)
• Improve: Tighten safety profiles (6eee0a9)
• Fix: Serial match verification mid-character (d927ea7)
• Improve: New regression tests for ligatures (29e9cd2)
• Fix: Compile-time Ice-Lake dispatch (7dd6209)
• Docs: Describe problematic chars (368bf5d)
• Improve: New probe refinement & tail verification (d9ceb8c)
• Improve: Detect more danger zones (d327cf8)
• Improve: Better fuzzing for substring search (9ad155e)
• Improve: Case folding variables naming (4ee616c)
• Fix: Case-folding around Glagolitic E2 ranges (68cd557)
• Improve: Fuzzing case-folding equivalene (4dcbe62)
• Fix: Check for incomplete set of 3-byte chars in case_fold_ice (ad036ef)
• Fix: Match new reusable needle ABI in Rust & Python (fa30741)
• Fix: Folding Greek final sigma in AVX-512 (bda1321)
• Fix: Handling Micro sign and Armenian ligatures (39516b3)
• Improve: Deduplicate body/tail kernel logic (d9e2409)
• Fix: Pass stress-tests under 10x multiple (1e050ad)
• Improve: Share abstractions for match validation (55c7c92)
• Fix: Cleaner script-specific window tracking (5a1ba33)
• Improve: Case-insensitive test coverage (8b2385d)
• Improve: Propagate metadata between queries (e3a6bb6)
• Fix: Detecting bicameral chars on Ice Lake (65c6c98)
• Improve: Faster test suite (74060ed)
• Fix: Classifying Armenian as bicameral (0ab964e)
• Improve: Test case-insensitive search against fold+find (aabb45e)
• Fix: 's' removed from the ASCII path (524ec7b)
• Docs: Policy for historical S sign 'ſ' (U+017F) (2226e25)
• Improve: Test coverage for case-insensitive search (a9a7d85)
• Fix: Mostlly passing tests (24e43aa)
• Improve: Simpler design for Ice-Lake case-insensitive search (9ce5a6f)
• Fix: Match new rune safety profiles (b313a8c)
• Fix: 'k' and C6 policy for Vietnamese (8974271)
• Improve: New safety profiles (afa24f7)
• Fix: Steping logic for safe slices under 16 bytes (b66ebd4)
• Improve: Default safe-window selection (1588a2d)
• Fix: Multiplication/division signs on Vietnamese path (5e0de6c)
• Fix: Special Cyrillic folding cases (fb09351)
• Improve: Print env settings at start (b4e269e)
• Fix: Serial fallback for archaic Polytonic Greek chars (659c2c7)
• Fix: Remove Ligature detection from the hot path (351236c)
• Fix: Mask offsets and Latin-A/B extensions (6aa2893)
• Fix: Using enum masks for character safety profiles (f30fb20)
• Improve: Uniform function naming (e751fbc)
• Improve: Uniform logic for case-insensitive search (a534597)
• Improve: Check "safe windows" even for small needles (b56ee54)
• Improve: New safety profiles for Unicode scripts (a6d75b9)
• Fix: Stale folded rune state (32b3df4)
• Improve: Cleaner Ice Lake kernels (4c6cf68)
• Fix: Stale pending_idx in fast ASCII iterator (8614658)
• Fix: Missing Strs.tape accessors (718f9c1)
• Fix: Named args in hmac_sha256 (29c5732)
• Docs: Missing API coverage (217055e)
• Improve: Log running tests (473d9db)
• Docs: Explore more scripts with examples (01f493b)
• Improve: 40% smalles fast-path selector state (3d6b244)
• Improve: Reuse needle anomalies logic (094eba8)
• Improve: Test more scripts (917bc03)
• Fix: Passing case-insensitive tests (9d5b9e3)
• Improve: Cleaner case-insensitive fuzzing (ddc8b7a)
• Improve: Unnecassary checks in ci-find (c19f11c)
• Fix: Case-insensitive search passes test (726bbbd)
• Improve: Case-insensitive search for Ry, Vi, El, Hy (Am) (fd89a88)
• Improve: Vietnamese fastt case-folding path (025b36d)
• Improve: Self-equality & overflow protection (b325b1c)
• Docs: Small-string safety comments (83ca417)
• Improve: Length-returning small-string API (be9e2d7)
• Improve: Avoid modulo division (024f677)
• Improve: Use ring-buffers for O(1) prefix hashes (2db5e54)
• Docs: Missing table info (c8b6ae1)
• Improve: Cleaner Raita kernels - unstable (f642fa9)
• Improve: Faster LUT on Ice Lake and Zen4+ (b13aef4)
• Improve: Avoid UTF-8 checks in case-fold (d8aac4a)
• Improve: Faster serial baselines for ASCII needles (e5227ad)
• Improve: Optional start/end for folded find (62ad6f7)
• Improve: Faster optional UTF-8 validation (7edba6f)
• Fix: Folding "中ABC" on Ice Lake (20dbef3)
• Improve: Boundary condition fold tests (bbea84f)
• Docs: Exaplain convoluted control-flow (44b6279)
• Fix: Gracefully handle Unicode spec download issues (44412bf)
• Fix: Require continuous substitution matrices (20ac49a)
• Make: Ignore UV lock (2c3d35d)
• Docs: Inconsistent UTF-8 fold explanations (c7a3012)

Release: v4.4.2 [skip ci]

Patch

• Fix: Windows MSVC compilation (#292) (70b20e2)

Added sz_at_least(n) macro for C99's static array parameter syntax, enabling compile-time bounds checking on fixed-size array arguments. In C mode, Clang will now warn when passing undersized arrays to annotated functions. The macro expands to nothing in C++ for compatibility.

// Compiler can now warn if the digest buffer is smaller than 32 bytes
void sz_sha256_state_digest(..., sz_u8_t digest[sz_at_least(32)]);

// Lookup tables must be at least 256 bytes
void sz_lookup(..., char const lut[sz_at_least(256)]);

See LWN.net article for background on this feature and its use in the Linux kernel.

Patch

• Improve: Harden C API with static n arrays (#289) (039c4b4)

To my knowledge, this is the first ever properly vectorized case-folding (aka .to_lower()) implementation compliant with Unicode (v17) and using SIMD (AVX-512 for Intel Ice Lake and newer). The results are remarkable across most languages, but it wasn't trivial to achieve. Unlike dense linear algebra workloads, such as in SimSIMD, no shared logic holds across all languages and code points here. After all, Unicode began in 1989 and covers languages and writing systems that took thousands of years to develop and decades to be organized into a standardized set of rules.

This implementation focuses on locale-independent conversion. It covers every one of 1000+ character folding rules in CaseFolding.txt of the Unicode spec, including:

• simple cases, like ASCII English letters: 'A' → 'a'.
• complex Latin extensions, where one codepoint expands into multiple characters: 'ẞ' → "ss".
• ligatures and mathematical symbols, like 'ffi' → "ffi".
• less common bicameral alphabets, including Armenian, Georgian, Vietnamese, and others.
• fast memcpy-like paths for unicameral scripts, like Chinese, Japanese, and Korean.

To benchmark all of those, I've extended the StringWars benchmarks with a new bench_unicode.rs and bench_unicode.py scripts and the bench_unicode.md report produced for two dozen datasets pulled from the Leipzig Wikipedia corpora. On most languages the performance is great, except for Georgian and Vietnamese for now:

Language	Standard 🦀	StringZilla 🦀		Standard 🐍	StringZilla 🐍
English 🇬🇧	482 MB/s	7.53 GB/s	16x	257 MB/s	3.14 GB/s	12x
German 🇩🇪	432 MB/s	2.59 GB/s	6x	260 MB/s	1.81 GB/s	7x
Russian 🇷🇺	217 MB/s	2.20 GB/s	10x	470 MB/s	1.56 GB/s	3x
French 🇫🇷	346 MB/s	1.84 GB/s	5x	274 MB/s	1.37 GB/s	5x
Greek 🇬🇷	220 MB/s	1.00 GB/s	5x	431 MB/s	779 MB/s	2x
Armenian 🇦🇲	223 MB/s	908 MB/s	4x	470 MB/s	746 MB/s	2x
Vietnamese 🇻🇳	265 MB/s	352 MB/s	1x	340 MB/s	291 MB/s	1x
Arabic 🇸🇦	232 MB/s	1004 MB/s	4x	467 MB/s	1.80 GB/s	4x
Bengali 🇧🇩	314 MB/s	6.17 GB/s	20x	694 MB/s	2.91 GB/s	4x
Chinese 🇨🇳	325 MB/s	1.21 GB/s	4x	697 MB/s	886 MB/s	1x
Czech 🇨🇿	322 MB/s	827 MB/s	3x	292 MB/s	688 MB/s	2x
Dutch 🇳🇱	471 MB/s	4.73 GB/s	10x	262 MB/s	2.97 GB/s	11x
Farsi 🇮🇷	235 MB/s	858 MB/s	4x	475 MB/s	1.42 GB/s	3x
Georgian 🇬🇪	294 MB/s	192 MB/s	1x	689 MB/s	488 MB/s	1x
Hebrew 🇮🇱	233 MB/s	1.01 GB/s	4x	473 MB/s	1.86 GB/s	4x
Hindi 🇮🇳	293 MB/s	6.32 GB/s	22x	682 MB/s	3.14 GB/s	5x
Italian 🇮🇹	439 MB/s	2.29 GB/s	5x	268 MB/s	1.93 GB/s	7x
Japanese 🇯🇵	330 MB/s	3.51 GB/s	11x	726 MB/s	2.00 GB/s	3x
Korean 🇰🇷	314 MB/s	861 MB/s	3x	623 MB/s	2.80 GB/s	4x
Lithuanian 🇱🇹	352 MB/s	864 MB/s	2x	274 MB/s	728 MB/s	3x
Polish 🇵🇱	364 MB/s	939 MB/s	3x	277 MB/s	786 MB/s	3x
Portuguese 🇧🇷	395 MB/s	2.38 GB/s	6x	270 MB/s	1.79 GB/s	7x
Spanish 🇪🇸	414 MB/s	2.38 GB/s	6x	272 MB/s	1.80 GB/s	7x
Tamil 🇮🇳	306 MB/s	6.05 GB/s	20x	712 MB/s	3.03 GB/s	4x
Turkish 🇹🇷	326 MB/s	852 MB/s	3x	284 MB/s	706 MB/s	2x
Ukrainian 🇺🇦	217 MB/s	2.09 GB/s	10x	476 MB/s	1.58 GB/s	3x

For a complete comparison, go to StringWars 😉

Minor

• Add: Fast path for Georgian case-folding (fa7422c)
• Add: Case-insensitive ops for Python (d88e30a)
• Add: Dispatch case-insensitive search (4ae91c0)
• Add: Serial case-insensitive find & compare (4b18f05)

Patch

• Fix: Eszett hex parsing warnings in Clang (8b27080)
• Fix: Avoid __builtin missing on MSVC (fdc95f3)
• Fix: Uninitialized values warning (b84c83e)
• Improve: Safer & faster case-folding on Ice Lake (bcd5d16)
• Improve: Case-folding on Ice Lake (bb23b60)
• Fix: Move Ice Lake kernels out of Haswell scope (b7cc2c4)
• Improve: Rename functions towards utf8_case* (44fbb92)
• Improve: Faster serial Unicode folding (aa1b21b)
• Improve: Re-group folding by char-length (c3586e2)
• Docs: Avoid locale-specific Unicode rules (333a778)
• Docs: Emoji-free doc section titles (#284) (dc11b40)

On AMD Zen5 Turin CPUs on different datasets, StringZilla provides the following throughput for splitting around whitespace and newline characters on 5 vastly different languages. Chinese and Korean texts, for example, are both made of mostly 3-byte letters, but Korean uses a lot of whitespace characters for syllable separation, while Chinese doesn't use any. French and English both use a lot of single-byte whitespace characters, but French uses many accented letters that are 2-byte long in UTF-8.

Library	English	Chinese	Arabic	French	Korean
Split around 8 newline combinations:
stringzilla::utf8_newline_splits	15.45 GiB/s	16.65 GiB/s	18.34 GiB/s	14.52 GiB/s	16.71 GiB/s
stdlib::split(char::is_unicode_newline)	1.90 GiB/s	1.93 GiB/s	1.82 GiB/s	1.78 GiB/s	1.81 GiB/s
Split around 25 whitespace characters:
stringzilla::utf8_whitespace_splits	0.82 GiB/s	2.40 GiB/s	2.40 GiB/s	0.92 GiB/s	1.88 GiB/s
stdlib::split(char::is_whitespace)	0.77 GiB/s	1.87 GiB/s	1.04 GiB/s	0.72 GiB/s	0.98 GiB/s
icu::WhiteSpace	0.11 GiB/s	0.16 GiB/s	0.15 GiB/s	0.12 GiB/s	0.15 GiB/s

On Apple M2 Pro:

Library	English	Chinese	Arabic	French	Korean
Split around 8 newline combinations:
stringzilla::utf8_newline_splits	5.69 GiB/s	6.24 GiB/s	6.58 GiB/s	6.70 GiB/s	6.29 GiB/s
stdlib::split(char::is_unicode_newline)	1.12 GiB/s	1.11 GiB/s	1.11 GiB/s	1.11 GiB/s	1.13 GiB/s
Split around 25 whitespace characters:
stringzilla::utf8_whitespace_splits	0.57 GiB/s	2.45 GiB/s	1.18 GiB/s	0.61 GiB/s	0.92 GiB/s
stdlib::split(char::is_whitespace)	0.59 GiB/s	1.16 GiB/s	0.99 GiB/s	0.63 GiB/s	0.89 GiB/s
icu::WhiteSpace	0.10 GiB/s	0.16 GiB/s	0.14 GiB/s	0.11 GiB/s	0.14 GiB/s

Minor

• Add: UTF-8 case-folding placeholders (15bcc43)
• Add: UTF-8 serial case-folding (65b652f)
• Add: SVE2 kernels for UTF-8 (d4504be)
• Add: Skip-ahead UTF-8 iterator interface (958be10)
• Add: NEON UTF-8 tokenization kernels (0259f58)
• Add: try_replace_all for Rust (35ed227)
• Add: NEON UTF-8 placeholders (f1fcdc5)
• Add: Lazy UTF-8 views for Rust (c08dc0c)
• Add: sz_utf8_unpack_upto64 for iterators (3ea1857)
• Add: UTF-8 length counting 15x faster (49d9da0)
• Add: utf8.h for new valid and find_nth interfaces (e0465d5)
• Add: UTF-8 bound checks for Rust (e7b4b9e)
• Add: UTF-8 boundary detection (f1e5318)

Patch

• Make: SZ_ENFORCE_SVE_OVER_NEON=0 by default (da5687d)
• Improve: Fewer loads in SVE2 and no fast paths (a06583a)
• Make: Bump macOS-13 → 15 in CI (98b8802)
• Improve: Fewer registers for e280xx masks in SVE2 (5434ebf)
• Improve: Faster SVE2 & Neon logic (bd9ddf5)
• Fix: NEON whitespace & newline equivalence (016c44a)
• Improve: UTF-8 equivalence checks (786a322)
• Fix: Missing i8 greater-than in AVX2 (dd4c4b0)
• Fix: MSVC-compatible uint8x16_t init (97cf851)
• Improve: Consistent var. names in UTF-8 tokenizers (5c6a32a)
• Fix: Aligned state compilation in NEON (31e4c8b)
• Fix: Missing svcompact_u8 in SVE2 (302af92)
• Improve: Include SVE2 benchmarks (4f558e1)
• Fix: Incorrect literal bound for test input (5e0f3ea)
• Improve: skip_empty arg for Python compatibility (0279383)
• Improve: Consistent split-iterator across languages (07c4d1c)
• Improve: Case-folding bump from Unicode 16 to 17 (9daa2a7)
• Fix: UBSAN issues in hash.h (36fa527)
• Docs: On complexity of case-insensitive substring search (ac5cb2f)
• Make: Bump Rust deps & drop ICU (ebc4296)
• Improve: New case-folding ABI (82528a7)
• Make: Separate file for UTF-8 unpacking (567cf17)
• Improve: Check UTF-8 case-folding (bf0ff0d)
• Make: Deprecate current UTF-32 unpacking code (b2b96f4)
• Fix: Misplaced UTF-8 skip in StringZilla (b838127)
• Fix: svmatch-ing zero characters in SVE2 kernels (6f045aa)
• Improve: Use fewer registers in SVE2 code (e52f4a1)
• Fix: short implicit casts (00bacfc)
• Improve: Test CLRF corner cases (0edc81f)
• Improve: Faster utf8_count_neon w/out u64 unpacking in loop (b583fa8)
• Improve: Fast path for 1-byte whitespace in NEON (73da441)
• Fix: Compile-time AES/SHA dispatch for Apple (8c34baf)
• Improve: More UTF-8 whitespace tokenization tests (8bb0324)
• Fix: no_std builds and doctests (bb699e9)
• Improve: Test UTF-8 decoding ops (849bff2)
• Fix: Out of bounds access in sz_sha256_*_ice (2bceb8d)
• Make: Correct env fields for .vscode/tasks.json (dda7704)
• Improve: Unlimited chunk size for UTF-8 iterators (aad09a4)
• Make: Tune Rust analyzer to use less RAM (ced9636)
• Fix: Skip U+001C, U+001D, U+001E (aca0473)
• Improve: Avoid optimization in more benchmarks (f979ed9)
• Improve: Fast path for UTF-8 whitespaces (a3c407f)
• Make: Build just 1 target for VS Code debug (26b0074)
• Fix: Signed comparisons for UTF-8 boundaries (f532ea2)
• Make: Redefining SZ_DEBUG=0 in CMake (febbdac)

Release: v4.2.3 [skip ci]

Patch

• Fix: Missing bounds checks in Rust (#273) (5219a4d)
• Fix: Type-casting UBs of movemask bitsets (7c42b98)
• Fix: Handling a larger order array (32b6350)
• Fix: head_length is pre-decremented to zero (1c5c7e8)
• Fix: Avoid std::enable_if for non-STL builds (568d90c)
• Fix: Lifetime of temp strings in ranges (73ce811)

Release: v4.2.2 [skip ci]

Patch

• Improve: LUTs in SVE (3d886d3)
• Make: Linux cross-compile matching Release CI (524b0d7)
• Fix: Check for Arm Neon support on windows (30320b7)
• Make: Removed pyarrow from windows arms python tests (eab8c3c)
• Make: Exclude KERNEL32.dll from stringzilla_bare checks (9edb804)
• Make: Disabled SVE when using MSVC (04c985b)
• Make: Use correct arch on windows for stringzillas/cuda (3fcd947)
• Make: Updated target arch for windows tests. (e6460e1)
• Fix: Disable windows min/max macros (00e902f)
• Fix: Replace processthreadsapi.h with windows.h (f09e4f9)
• Make: Expand CMAKE_HOST_SYSTEM_PROCESSOR (fe09f8d)
• Make: Revert --sysroot cross-compile commands (579c82d)
• Fix: Accessing ARM64_CNTVCT on Windows (5e6777d)
• Make: Avoid redefining arch=armv8.2-a in pragmas (636147d)
• Make: Expand CMAKE_HOST_SYSTEM_PROCESSOR (1f90f6c)
• Make: Link to libc++ in LLVM builds on MacOS (1c8b29b)
• Make: Revert _M_ARM64=1 flags for MSVC (25311a6)
• Make: Enable Posix extensions for Python builds (9fe4f7c)
• Make: Missing macros for winnt.h(169) C1189 error (8ef98a9)
• Fix: Reading mrs w/out inline Asm on MSVC (d804c9f)
• Make: Override --sysroot for "Cross Compile" builds (d3d901d)
• Make: Use valid arch flags on MSVC (5aba122)
• Make: Cross compile checks now correct for MSVC (7664f67)
• Make: Windows arm now uses the correct compiler (7c2e9a0)
• Make: cmake set ARCHIVE_OUTPUT_DIRECTORY to binary dir (f1ec210)
• Make: Use ninja for windows deploy builds (0af43c8)
• Make: Fixed Windows deploy (8ff2ad7)
• Make: Include experimental Arm cross-compilation (4d86312)

Exposing SHA-256 to GoLang was tricky. Clang worked fine. GCC failed. It turned out that GCC was too shy about inlining my code, resulting in excessive stack space usage... Now, JavaScript, Swift, and GoLang bindings all support incremental SHA-256 procedures 🥳. Thanks to @MarekKnapek for reducing the stack memory usage of the serial SHA variant!

Moreover, thanks to @laurenspriem for highlighting the SIGILL when probing ID registers on older Arm CPUs. I've now guarded first mrs probes with signal handlers. Ugly solution, but it may work 😅 I've also improved the capability detection code on Arm-based Windows machines, using the OS-specific <processthreadsapi.h> functionality, so now not only pure NEON, but also NEON+SHA+AES kernels, should be dispatched just fine!

Thanks to @ashbob999, StringZilla is also getting more stable Windows builds and stringzilla_bare coverage in our CI 🦺

Patch

• Make: Removed rand/free/malloc stubs when avoiding libc (0148282)
• Make: Deploy stringzilla_bare for windows (e4ddce8)
• Make: Added .lib file to uploaded windows archives (2dc6936)
• Make: Add MSVC bare builds back (5cc5f01)
• Make: Added stringzilla_bare checks (bbc5cca)
• Fix: Avoid unused POSIX extensions on macOS (aeb06a5)
• Make: Deprecate old cross-compilation scripts (2f34c2d)
• Improve: Drop -pedantic for POSIX extensions (e99d557)
• Make: Pre-define CMake properties, like -lpthread and pointer size (7722bb1)
• Improve: serialize_capability for Ice Lake on Clang (58f8cf9)
• Make: Skip compiler checks for cross-compilation (60988f3)
• Fix: Unused capabilities in Arm macOS builds (511a09e)
• Docs: Listing ./scripts and StringWars (5af84dd)
• Make: Pass -D CMAKE_SYSROOT in cross-compiling CI (a26fc73)
• Fix: Suppress unused alloc warnings (4868d7f)
• Make: Reduce CMake nesting (dda024d)
• Make: Propagate cross-compilation settings (5070321)
• Improve: Detect NEON+SHA+AES via WinAPI (3b175f8)
• Fix: Probe mrs for avoid SIGILL on older Arm (d2f8e97)
• Fix: Isolate & skip SHA-256 tests in Go with GCC (0874b13)
• Fix: Deprecates sz_checksum (97f9ecf)
• Make: More aggressive inlining (e8f33c1)
• Make: Uniform hardware specs logging (f826dfc)
• Improve: Expose Capabilities to GoLang (5f2cc97)
• Improve: Branchless serial SHA-256 block processor (fe7efe2)
• Fix: Missing modulo in SHA #254 (5a513b7)
• Improve: Smaller stack usage in SHA-256 (#253) (a298be0)
• Fix: No noescape/nocallback for stateful hashes (f8d321f)
• Fix: Violating u32/u64 aliasing (7e55e5c)
• Fix: Missing SSE flags for SHA (403b28b)
• Improve: io.Writer & hash.Hash64 interface for Go (05f89ca)
• Improve: Expose sz_dispatch_table_init for Go (5ff7ba1)
• Fix: Missing Goldmont & Ice SHA dispatch (e29bded)
• Fix: Supporting unaligned SHA-256 states (c770e48)
• Fix: Missing C.sz_checksum (652735d)
• Fix: Hex formatting in Swift on Linux (fc65328)
• Improve: SHA for Go, JS, Swift (a165322)

User-facing updates:

• 🆕 SHA-256 checksums
• 🆕 Detect compilation settings

Implementation details:

• 🆕 Intel Goldmont capabilities level
• 🆕 Arm NEON+SHA capabilities level
• Hardened Rust builds & capability masking
• Faster buffer filling in sz_hash in NEON backend
• Fixed tail handling in sz_copy in SVE backend

Minor

• Add: Check comp-time capabilities (3347be4)
• Add: sz_cap_goldmont_k capability! (f70e927)
• Add: neon+sha new capability! (fcb68a4)
• Add: Sha256 to bench_token (bb077da)
• Add: hmac_sha256 APIs (bf1971e)
• Add: Sha256 class for Python (6ae7b75)
• Add: Initial Sha256 variant for NEON (bd35030)
• Add: SHA256 for Arm (20672dd)

Patch

• Fix: Avoid unaligned SHA loads on ArmV7 (ebf0503)
• Fix: Sign conversion warning (3c3e5fc)
• Make: before-all for dnf on Fedora & apt on Debian (fc74452)
• Make: Consume env-vars for Rust backend builds (222fc39)
• Improve: Amortize bench_unary costs (d8d19ce)
• Fix: Init uint32x4_t on MSVC (3dce631)
• Improve: Bring back SVE2 hash for short inputs (b1c750b)
• Improve: More sorting tests (61e08ce)
• Improve: Simplify SVE memory-ops (35e2236)
• Fix: sz_copy_sve tail issue (2fa818d)
• Fix: Avoid <arm_neon_sve_bridge.h> (e5b4496)
• Improve: Different SHA pipeline for AArch64 (c8aafd3)
• Improve: Try better SHA pipelining (313f71f)
• Improve: Faster 2-block SHA256 on NEON (9425341)
• Improve: Deprecate SVE2 hashing (9cb1588)
• Improve: Try using non-temporal SVE loads (4572a63)
• Fix: svlasta_u64(svpfalse_b()) UB (4833e83)
• Improve: Westmere-like hash updates in NEON (064355f)
• Improve: Hardening Rust builds (d6a9ba6)
• Fix: Type-casting on Arm (03a0340)

Thanks to @Algunenano and the broader ClickHouse team for help, back-porting StringZilla kernels to older CPUs 🤗 With this release:

• Substring search and hashing on CPUs from Westmere to Haswell will become at least 2x faster.
• Inferring Skylake capabilities in dynamic dispatch won't require VAES extensions only needed for Ice Lake and newer.
• MSVC will correctly detect Haswell, Ice Lake, and NEON capabilities for compile-time dispatch, lacking options to differentiate other platforms from macros.

Minor

• Add: Nehalem (SSE4.2) CPU support (#247) (2c58385)

Patch

• Fix: Checking for SSE and AES in MSVC (f7d95ad)
• Docs: More links & GPU mentions (04d6ae3)
• Improve: Reuse AES-NI since Westmere (a3b7cd5)
• Fix: Replace Nehalem with Westmere (#249) (fe30683)
• Fix: VAES debuted in Ice Lake (8cef111)

Release: v4.0.15 [skip ci]

Patch

• Improve: Faster unaligned loads in fingerprints (30ad812)
• Fix: Avoid Ice Lake instructions on older CPUs (96ce576)
• Improve: Faster streaming hashes on x86 (2ce62ba)
• Docs: Levenshtein wave shape (8e1f70c)

Release: v4.0.14 [skip ci]

Patch

• Fix: ODR violation for raise in C++ (#238) (6e826d7)

Release: v4.0.13 [skip ci]

Patch

• Make: Pull CUDA within CIBW_BEFORE_ALL (6316330)

This release fixes a critical bug where non-owning Strs slices incorrectly copied entire parent data during GPU memory allocation, instead of just the slice portion. The fix ensures proper Apache Arrow-compatible StringTape format handling with correct offset normalization for zero-copy operations. GPU memory management is now significantly more efficient, eliminating unnecessary re-allocations when data already resides in GPU memory through intelligent parent chain traversal.

A new stringzillas.to_device() function enables explicit GPU memory pre-allocation, useful for testing and performance optimization:

import stringzilla as sz
import stringzillas as szs

# Create strings and slices
strs = sz.Strs(["hello", "world", "test", "data"])
slice_view = strs[1:3] # Non-owning view of ["world", "test"]

# Pre-allocate on GPU (if available)
gpu_strs = szs.to_device(strs)
gpu_slice = szs.to_device(slice_view) # Correctly handles slice offsets

Cross-platform builds are now more stable with fixes for Windows ARM64 cross-compilation, ensuring mutually exclusive architecture flags prevent header conflicts. The CI/CD pipeline correctly generates stringzillas-cuda packages by properly propagating environment variables through cibuildwheel. Enhanced test coverage includes complex Unicode scenarios with RTL text, emoji sequences, and different normalization forms. Documentation has been extended with Rust examples showcasing zero-copy compute_into APIs using StringTape format.

Patch

• Make: Mutually exclusive platform flags (773d959)
• Fix: Skip to_device tests w/out GPUs found (d701592)
• Make: Propagate SZ_TARGET into CIBW env (b222e82)
• Improve: Avoid realloc for on-GPU views (77e67cf)
• Docs: Zero-copy Rust compute_into API with StringTape (f4ad81e)
• Improve: Validate to_device(Strs) for unicode (f3c5357)
• Improve: Pre-send to GPU with to_device (c78cd21)
• Fix: Same Strs slicing as in StringTape (b4f8d12)

Release: v4.0.11 [skip ci]

Patch

• Improve: Striping APIs for Python (75321c5)
• Docs: Coloring StringZilla green! (c3ccfa6)

Release: v4.0.10 [skip ci]

Patch

• Fix: Propagate big-endian & SIMD flags (0d6c4ef)

Release: v4.0.9 [skip ci]

Patch

• Make: Stricter SZ_IS_64BIT checks (0edabf5)
• Make: Revert PyPI trusted publishing (c679e41)
• Fix: Unmark__i386__ & _M_IX86 as 64-bit (4f3d8dc)
• Make: Better detect 64-bit Py builds (1b91246)
• Make: Identical CI job names (d15e779)

Release: v4.0.8 [skip ci]

Patch

• Docs: Outdated algorithm details (68dc092)
• Make: Guess platform for PyPI and sdist (e1966de)
• Make: Embed SZ_TARGET.env for PyPI sdists (57822a4)
• Make: Require serial package for parallel PyPI packages (d3ef8c8)
• Make: Move Py benchmarks to StringWa.rs (52c90da)
• Improve: Compare Levenshtein to CuDF (fe1e32b)
• Make: Bump StringTape (da974f5)
• Improve: Take slices in compute_into (b507a8f)
• Improve: compute_into APIs for Rust (103019c)
• Docs: Ship different __description__s (1467001)
• Make: Supress warnings in Windows CIBW (a879dd0)
• Make: Drop Windows before-test override (d19e2fa)
• Improve: Skip sz PyTests in szs runs (8abcdab)
• Fix: Windows compilation issues (fee2ffc)
• Fix: Safer conversion to RawParts (a9cfa22)

Release: v4.0.7 [skip ci]

Patch

• Fix: Use unsigned offsets & smaller tapes (71f6069)
• Improve: Clippy's static Rust suggestions (7282fc2)
• Fix: Revert zero-copy Rust interfaces (c12a9c8)
• Fix: Visibility warnings (6dc2412)
• Fix: Input traits & docs consistency (3bf13da)
• Improve: Support zero-copy tape inputs in Rust (ad40e6a)
• Make: Allow partial wheel uploads to PyPI (5dacaa9)

Release: v4.0.6 [skip ci]

Patch

• Fix: Move macros for MSVC visibility (e5c19ee)
• Make: Require macOS 10.13 for x86 (f136fd8)
• Make: Only pack include/ from Fork Union (948ae74)
• Fix: Missing void specialization for enable_if (225d55a)
• Fix: Can't use __attribute__ with MSVC (f2170d8)
• Make: Hide Fork Union crate info for packaging (fd3485c)

Release: v4.0.2 [skip ci]

Patch

• Make: Override get_requires_for_build_wheel (c83f1d3)

Release: v4.0.1 [skip ci]

Patch

• Make: Delay NumPy include path resolution (ca8186b)

This PR entirely refactors the codebase, separating the single-header implementation into separate headers. Moreover, it brings faster kernels for:

• Sorting of string sequences and pointer-sized integers 🔀
• Levenshtein edit distances for fuzzy matching of UTF-8 or DNA 🧬
• Needleman-Wunsch and Smith-Waterman scoring, see affine-gaps ☣️
• Multi-pattern sketching and fingerprinting on GPUs 🔍
• AES-based portable general-purpose hashing functions #️⃣

And more community contributions:

• Detecting CPU capabilities 👏 @GoWind - #196
• Windows cross-compilation 👏 @ashbob999 - #169
• CMake refactor 👏 @friendlyanon - #85
• Charset initialization 👏 @alexbarev - #200
• Benchmarking sorting algorithms 👏 @ashbob999 #209
• Allocator initialization and unknown pragmas on MSVC 👏 @GerHobbelt #231
• Big-endian SWAR substring-search backends 👏 @SammyVimes #75
• NodeJS and GoLang bindings groundwork 👏 @MarkReedZ #151
• New C++23 APIs 👏 @PleaseJustDont #225
• Repeated help with Rust 👏 @mikayelgr and @grouville #215

Huge thanks to our partners at Nebius for their continued support and endless stream of GPU installations for the most demanding computational workloads in both AI and beyond!

Why Split the Files? Matching SimSIMD Design

Sadly, most modern software development tooling is subpar. VS Code is just as slow and unresponsive as the older Atom and the other web-based technologies, while LSP implementations for C++ are equally slow and completely mess up code highlighting for files over 5,000 Lines Of Code (LOCs). So, I've unbundled the single-header solution into multiple headers, similar to SimSIMD.

Also, similar to SimSIMD, CPU feature detection has been reworked to separate serial implementations, Haswell, Skylake, Ice Lake, NEON, and SVE.

Faster Sequence Alignment & Scoring on GPUs

Biology is set to be one of the driving forces of the 21st century, and biological DNA/RNA/protein data is already one of the fastest-growing data modalities, outpacing Moore's Law. Still, most of the BioInformatics software today is flawed and pretty slow. Last year, I helped several BioTech and Pharma companies scale up their data-processing capacity with specialized optimizations for various niche use cases. Still, I also wanted to include baseline kernels for the most crucial algorithms to StringZilla, covering:

• Levenshtein distance computation for binary & UTF-8 data for general-purpose fuzzy string matching on CPUs and GPUs,
• Needleman-Wunsch and Smith-Waterman global and local sequence alignment with both linear and affine Gotoh gap extensions, so common in BioInformatics, on both CPUs and GPUs.

These kernels are hardly state-of-the-art at this point, but should provide a good baseline, ensuring correctness and equivalent outputs across different CPU & GPU brands.

Faster Sorting

Our old algorithm didn't perform any memory allocations and tried to fit too much into the provided buffers. The new breaking change in the API allows passing a memory allocator, making the implementation more flexible. It now works fine on both 32-bit and 64-bit systems.

The new serial algorithm is often 5 times faster than the std::sort function in the C++ Standard Template Library for a vector of strings. It's also typically 10 times faster than the qsort_r function in the GNU C library. There are even quicker versions available for Ice Lake CPUs with AVX-512 and Arm CPUs with SVE.

Faster Hashing Algorithms

Our old algorithm was a variation of the Karp-Rabin hash and was designed more for rolling hashing workloads. Sadly, such hashing schemes don't pass SMHasher and similar hash-testing suites, and a better solution was needed. For years, I have been contemplating designing a general-purpose hash function based on AES instructions, which have been implemented in hardware for several CPU generations now. As discussed with @jandrewrogers, and can be seen in his AquaHash project, those instructions provide an almost unique amount of mixing logic per CPU cycle of latency.

Many popular hash libraries, such as AHash in the Rust ecosystem, cleverly combine AES instructions with 8-bit shuffles and 64-bit additions. However, they rarely harness the full power of the CPU due to the constraints of Rust tooling and the complexity of using masked x86 AVX-512 and predicated Arm SVE2 instructions. StringZilla does that and ticks a few more boxes:

• Outputs 64-bit hashes and passes the SMHasher --extra tests.
• Is fast for both short (velocity) and long strings (throughput).
• Supports incremental (streaming) hashing, when the data arrives in chunks.
• Supports custom seeds for hashes and has it affecting every bit of the output.
• Provides dynamic-dispatch for different architectures to simplify deployment.
• Documents its logic and guarantees the same output across different platforms.

Implementing this logic, which provides both fast and high-quality hashes, often capable of computing four hashes simultaneously, made these kernels handy not only for hashing itself, but also for higher-level operations like database-style hash joins and set intersections, as well as advanced sequence alignment algorithms for bioinformatics.

Fingerprinting, Sketching, and Min-Hashing... using 52-bit Floats?!

Despite deprecating Rabin-Karp rolling hashes for general-purpose workloads, it's hard to argue with their usability in "fingerprinting" or "sketching" tasks, where a fixed-size feature vector is needed to compare contents of variable-length strings. The features should be as different as possible, covering various substring lengths, so polynomial rolling hashes fit nicely!

That said, implementing modulo-arithmetic over 64-bit integers is extremely expensive on Intel CPUs:

• VPMULLQ (ZMM, ZMM, ZMM) for _mm512_mullo_epi64:
  • on Intel Ice Lake: 15 cycles on port 0.
  • on AMD Zen4: 3 cycles on ports 0 or 1.
• VPMULLD (ZMM, ZMM, ZMM) for _mm512_mullo_epi32:
  • on Intel Ice Lake: 10 cycles on port 0.
  • on AMD Zen4: 3 cycles on ports 0 or 1.
• VPMULLW (ZMM, ZMM, ZMM) for _mm512_mullo_epi16:
  • on Intel Ice Lake: 5 cycles on port 0.
  • on AMD Zen4: 3 cycles on ports 0 or 1.
• VPMADD52LUQ (ZMM, ZMM, ZMM) for _mm512_madd52lo_epu64 for 52-bit multiplication:
  • on Intel Ice Lake: 4 cycles on port 0.
  • on AMD Zen4: 4 cycles on ports 0 or 1.

It's even pricier on Nvidia GPUs, but as you can see above, we may have a way out! x86 has a cheap instruction for 52-bit integer multiplication & addition. Moreover, 52 is precisely the number of bits we can safely use to exactly store an integer inside a double-precision floating-point number, opening doors to a really weird, but equally as implementation of rolling hash functions across CPUs and GPUs, implemented using 64-bit floats, and Barrett's reductions for modulo-arithmetics, to avoid division!

Nest Steps

• Ditch using constant memory in CUDA. The original design was to load the substitution tables for NW and SW into shared memory, but a simpler design with constant memory was eventually accepted. That proved to be a mistake. The performance numbers are way lower than expected.
• Use Distributed Shared Memory for 10x larger NW and SW inputs. That would require additional variants of linear_score_on_each_cuda_warp_ and affine_score_on_each_cuda_warp_, which scale to 16 blocks of (228 - 1) KB shared memory on each, totaling 3.5 MB of shared memory that can be used for alignments. With u32 scores and 3 DP diagonals (in case of non-affine gaps) that should significantly accelerate the alignment of ~300K-long strings. But keep in mind that cluster.sync() is still very expensive - 1300 cycles - only 40% less than 2200 cycles for grid.sync().
• Add asynchronous host callbacks for updating the addressable memory region in NW and SW.
• Potentially bring back stringzilla_bare builds on MSVC, if needed.

Major

• Break: Output error messages via C API (fc94890)
• Break: New wording for incremental hashers (66898ad)
• Break: Rust namespaces layout (b3338bb)
• Break: Refactor Strs ops (009b975)
• Break: Rename again (fb56b60)
• Break: Drop fingerprinting bench (3e04157)
• Break: sz::edit_distance -> Levenshtein (d44beb4)
• Break: C++ lookup and fill_random (1ce830b)
• Break: charset/generate -> byteset/fill_random (2ce2b49)
• Break: New calling convention in similarity.h (095bc2d)
• Break: Return error-codes in sort functions (944804e)
• Break: look_up_transform to lookup API (e0055d5)
• Break: checkum to bytesum, new hash, and PRNG (71f1f4b)
• Break: Pointer-sized N-gram Sorting (0c38bff)
• Break: sz_sort now takes allocators (ec81663)
• Break: Deprecate old fingerprinting (38014ee)
• Break: Replace char_set constructor with literals (2c49eae)

Minor

• Add: Str.count_byteset for Python (802d699)
• Add: GoLang official support (234b758)
• Add: HashMap traits for Rust (e555cc3)
• Add: try_resize_and_overwrite (e00a98b)
• Add: Hashers for Swift (be363c9)
• Add: Big-endian SWAR backends (607dd14)
• Add: Zero-copy JS wrapper for buffers (4bf2dd6)
• Add: sz.fill_random for Python (3565f9b)
• Add: Allow resetting the dispatch table (68172a0)
• Add: Ephemeral GPU executors if no device is passed (6431900)
• Add: Capabilities getters for SW (86c89b7)
• Add: StringZillas for Rust draft (48a1120)
• Add: Fingerprinting benchmarks (471649e)
• Add: On GPU fingerprints in Python (6f26629)
• Add: basic_rolling_hashers CUDA port (a25e3f2)
• Add: Unrolled fingerprinting backends (9a04c95)
• Add: NW and SW scoring classes (958c9e1)
• Add: SW scoring (3e8a3a6)
• Add: NW scoring (079a8c1)
• Add: Capability-constrained Py constructors (d0dfa0e)
• Add: LevenshteinDistancesUTF8 in Py (4cd6c7d)
• Add: Wrap DeviceScope for Python (30b8fd3)
• Add: Make Strs from lists, tuples, generators (05a5434)
• Add: StringZillas Python tests (978d13d)
• Add: Strs layout conversion tests (4e5df62)
• Add: Exportable _sz_py_api capsules (9d5935b)
• Add: Levenshtein kernels in shared lib (42a815f)
• Add: Strs.from_arrow conversion (0441b32)
• Add: Draft fingerprinting C binding (3f5e004)
• Add: Fingerprinting baselines (f997dad)
• Add: SZ_NOINLINE (4e6e1af)
• Add: Draft fingerprinting on GPUs (11af644)
• Add: Haswell rolling fingerprints (25d3ee6)
• Add: Draft exploration of float fingerprints (3955eee)
• Add: lock_guard to avoid STL (e6cdb93)
• Add: Parallel fingerprinting (4ccac06)
• Add: to_span, to_view helpers (4fb9283)
• Add: Min-Hashing basic_rolling_hashers (63447d3)
• Add: Fingerprinting benchmarks (00c68d5)
• Add: 64-bit double fingerprinting (9ed6006)
• Add: Test rolling hashes (a065aa0)
• Add: Fingerprinting drafts are back :) (9ab4a2b)
• Add: Draft parallel library backend (94147ed)
• Add: Long haystack CUDA kernels for find-many (51c9171)
• Add: find_many minimal counter & benchmarks (3c615d9)
• Add: bench_find_many.cpp (2d594e4)
• Add: CUDA Aho Corasick placeholder (d087afa)
• Add: Parallel Ice Lake variants (6aaf16c)
• Add: NW and SW for Hopper (e53c8d5)
• Add: Hopper Levenshtein kernels (cb7c48d)
• Add: Affine gaps Levenshtein on Kepler (f09bbf9)
• Add: Ice Lake Affine Levenshtein kernels (51afad5)
• Add: Affine Levenshtein variants on GPU (3e1df93)
• Add: sz_similarity_gaps_t enums (5b410b8)
• Add: Concepts & new multithreading scheme (6be0da5)
• Add: Draft StringCuZilla C API (1975351)
• Add: Draft executors for StringCuZilla (e1a37de)
• Add: C++20 concepts (f7b091c)
• Add: Unmasked Ice Lake Levenshtein (2b4fa09)
• Add: Affine Levenshtein-Gotoh variants (25ab3b6)
• Add: Draft global scoring in CUDA (07196d0)
• Add: Affine gap extensions baseline (945613c)
• Add: Draft device-wide similarity kernels (871d7bc)
• Add: Levenshtein on Kepler (938bf6f)
• Add: Parallel multi-needle search with OpenMP (c8dc314)
• Add: Draft parallel substring search (cf53b9b)
• Add: Overflow-risk error codes (783cfa8)
• Add: Random access ranges (0a26059)
• Add: safe_vector, safe_array (ac6218a)
• Add: Immutable iterators for Arrow tapes (b70096b)
• Add: indexed_container_iterator (a5ca2ac)
• Add: Multi-pattern exact substring search (a83cdb5)
• Add: NW benchmarks on GPU (1879aeb)
• Add: Multi-flag sz_caps (f76fa40)
• Add: Repeat similarity benchmarks on CPU (f64fc56)
• Add: gpu_specs and cuda_status_t (c36d2b8)
• Add: Ice Lake similarity kernels (3c8d181)
• Add: Fetching Nvidia GPU specs (d0bdd17)
• Add: New batch similarity benchmarks (aca3301)
• Add: Warp-shuffle optimizations (4a62715)
• Add: Mem consumption CUDA tests (53d3e0d)
• Add: New NW and SW GPU kernels (ca0fece)
• Add: Blosum62 and NUC.4.4 matrices (8a99373)
• Add: Separate parallel & serial tests (3cb8dd0)
• Add: Parallel SW & NW scoring in OpenMP (85b9675)
• Add: Thrust-like constant_iterator (6751e79)
• Add: arrow_strings_tape::try_append (6d7f221)
• Add: CUDA scoring benchmarks (7ac0fdd)
• Add: Baseline NW and SW alignment with ~O(NM) space (044c7cc)
• Add: Horizontal scoring in OpenMP (02a2481)
• Add: Local alignment adaptation (109d8de)
• Add: Parallel GPU kernels (2861a85)
• Add: OpenMP score_diagonally (427d5b5)
• Add: lookup transform in Rust (3090472)
• Add: OpenMP C++ draft (6f8cdb9)
• Add: Stateful hashing in Rust (763538e)
• Add: Expose find_byteset in Rust (667ea91)
• Add: Set intersections in Rust (06784fc)
• Add: Sorting in Rust (e467649)
• Add: New memory benchmarks (dfd6ddf)
• Add: Draft sz_find_sve (d7ede5d)
• Add: Faster find_byte with SVE (56a49c8)
• Add: New string similarity benchmarks (c12e6c4)
• Add: Short string hashing in SVE2 (a007c7c)
• Add: SVE & SVE2 bytesum (d6f87d7)
• Add: SVE2 macros (9fbdc9a)
• Add: Intersection benchmarks (e860af0)
• Add: SVE backend for sorting (148b615)
• Add: All new benchmarking suite (4744406)
• Add: Comparisons in SVE (c31020d)
• Add: Arm NEON hashing (4b3847d)
• Add: Missing SVE placeholder definition (63f0368)
• Add: C++ argsort, intersect (5ea0698)
• Add: status_t for errors in C++ (63daa5f)
• Add: Feature-extraction placeholder (de62723)
• Add: Intersections on Ice Lake (ea5dc76)
• Add: Serial JOINs (c7b841e)
• Add: Dispatched version API (9a32744)
• Add: Fetching dynamic library version in C (3538e97)
• Add: PRNG for Haswell & serial backend (6659aa0)
• Add: Streaming hashing on Ice Lake & Skylake X (2607d45)
• Add: Streaming hash benchmarks (8ac3a23)
• Add: Hashing on Haswell & Skylake-X (3c345bc)
• Add: Missing sz_sequence_t helpers (dc7c109)
• Add: Sorting placeholders & dispatch (cc98389)
• Add: sz_sequence_argsort_ice (69d4ecb)
• Add: AES-based hash placeholders (cb18c78)
• Add: Smaller Sorting Networks (cd6859a)
• Add: String sorting tests for different lengths (c670ccd)
• Add: Separate Skylake-X & Ice Lake checksums (554f50d)
• Add: New Levenshtein distance kernels (43471aa)
• Add: Missing Rust interfaces (1765f33)

Patch

• Docs: Pre-release stats update (c08c6c3)
• Make: Upgrade setuptools in CI (492ecc0)
• Improve: Allow RuntimeError for engine calls (f9cbb00)
• Docs: Section titles (755f583)
• Improve: PyTest invalid input arguments (f5e46d5)
• Improve: All new similarity-scoring benchmarks (3713e72)
• Make: Option to disable sanitizers for masked IO (f880621)
• Make: Default to Python 3.12 for better itertools (f2704d7)
• Fix: Naming scorers in Python like in Rust (d0bc604)
• Fix: Avoid SWAR on big-endian (230e354)
• Improve: More big-endian SWAR tests (7a4b78f)
• Fix: Drop old similarity APIs in benchmarks (230fc13)
• Make: Packaging for Py & Go (4c819d6)
• Improve: Byte-set counting PyTests (bd692d2)
• Docs: What to know about CUDA (15349c6)
• Improve: Str_like_* naming convention (5fdd8ee)
• Fix: Merge artifacts & lifetime annotations (c71e67e)
• Docs: Wording & AI dashes (4a06f3f)
• Make: Packaging for NPM (0d95f13)
• Make: Drop long-deprecated .releaserc (8b5af6e)
• Make: Enable SIMD in NodeJS builds (c8f6a49)
• Improve: Polish parallel string test names (61cc860)
• Make: Reuse SIMD compilation flags in build.rs (d129f95)
• Make: Missing AES definitions for lib builds (e951c4d)
• Docs: Hashing sections for each SDK (cb4fe1b)
• Improve: Expose .capabilities to JS (c557a62)
• Improve: Test incremental hashers (7c6d37d)
• Fix: Self-move construction of basic_string (5fb0e74)
• Fix: Strict aliasing violation (a5a2421)
• Fix: __cpp_lib_string_resize_and_overwrite test guards (cb2d6a8)
• Docs: How to use parallel algorithms (ecad156)
• Fix: Stricter following of SZ_AVOID_STL (c3040b7)
• Docs: JS Quick Start (c923ed2)
• Make: Publish to NPM (b2cedc4)
• Make: Drop CodeQL noise (7c9caac)
• Docs: Describe dynamic dispatch & linking (e6f2c02)
• Improve: NodeJS groundwork & corner-case tests (#151) (00d75f5)
• Fix: _MSC_VER to __GNUC__ conditions (83492ac)
• Fix: Unknown pragmas in MSVC (#231) (78bbc11)
• Make: Parallel algorithms CI/CD for PyPI (9b8c466)
• Make: Ignore formatting blames (b69206f)
• Make: JSON & YAML uniform formatting (320bddd)
• Make: Explicit CodeQL coverage in CI (e9fb38d)
• Fix: Match new C-level DeviceScope behavior (f217b86)
• Fix: Sorting on big-endian s390x (8d2d9c8)
• Fix: Ruff-statically suggested issues (edaff0e)
• Improve: Disable E722 import exception warning (3aedfb2)
• Fix: Check for immutable Py buffers (e8c437e)
• Improve: Test PRNG in Py and boundary Strs sizes (3d76e8f)
• Fix: np.random v1 vs v2 compatibility also in szs. (5f8ac03)
• Fix: Prevent PyTest from parsing invalid UTF-8 (30c8935)
• Fix: Don't repeat seed-ed fuzzy tests (efceb0b)
• Fix: np.random v1 vs v2 compatibility (aea096b)
• Make: Forwarding SZ_IS_QEMU_ (d433db0)
• Fix: Minor logical inconsistencies & unused vars (679f7b9)
• Improve: Disable SVE in QEMU runs (072ee2e)
• Fix: Avoid sys.getrefcount tests on PyPy (80fa90e)
• Improve: Check rich comparisons before sorting (a21c44d)
• Improve: Session-scope fixture for PyTest env logs (3d84812)
• Improve: Fuzz PyTests and log environment (6545a04)
• Make: Respect env-vars for -arch (6a450f6)
• Make: Upgrade GitHub actions (dec93d0)
• Make: Avoid universal builds defaults for pip install . (c462f42)
• Improve: Guard compiler pragmas (42ad14d)
• Make: Bump FU to avoid missing +wfxt target (337257b)
• Make: Detect CPU AES support on Arm (ae74d44)
• Make: Reinstall pre-packaged CMake on macOS-14 (24967c7)
• Fix: Fall-back CPU alloc for fingerprints (b582d5d)
• Make: Drop macOS Universal builds (e5cfb08)
• Fix: Sorting difference on 32/64 bit machines (0e17a3d)
• Make: Respect MACOSX_DEPLOYMENT_TARGET (95a95fe)
• Make: Avoid fail-fast for Python pre-release wheels (a7c3f04)
• Fix: Win32 compilation issues (f792366)
• Improve: Test against Affine Gaps (65d323e)
• Improve: Avoid many unified memory re-allocs (d0db2d4)
• Fix: Intersect scopes HW capabilities (e4245b7)
• Make: Drop Python 3.7, require 3.8+ (c65bf5e)
• Make: Verbose PyTest logging in CI (c273d52)
• Fix: OS-feature-gate AVX checks (cd81b4a)
• Fix: Linking to 64-bit symbols (7da4dac)
• Make: Log HW caps in CI before tests (2c02d5b)
• Fix: Type-casting on MSVC (a347dcd)
• Make: Irrelevant links & comments (d6221e6)
• Make: Target Hopper 90a in Py & C (c317280)
• Make: Outdated 64-bit detection envs (72d0f4b)
• Docs: Wording typos (cf1623c)
• Make: Missing affine-gaps dep (fa52363)
• Docs: No Alpine flow on release CI (f214169)
• Fix: Argument order (10dff6c)
• Make: Can't read SWIFT_VERSION for container (a4015f1)
• Improve: Differentiate capabilities_mode in PyTest (2bee1e4)
• Fix: Dispatch serial code for bytes_per_cell <= 2 (48b4406)
• Improve: New error codes for CPU/GPU interop (6e597ca)
• Fix: Avoid UB assigning i8x256x256 matrix (7ac3b83)
• Make: Bump FU due to sign conversions warnings (cbd160a)
• Fix: Detect missing GPUs at runtime (0bc0f8a)
• Improve: Formatting Swift (3d6669c)
• Improve: Test against affine-gaps (48a538c)
• Improve: Reduce sign-casting issues (a340131)
• Fix: Check CUDA in szs_capabilities (42f043c)
• Improve: Build & test reproducibility (446e14e)
• Make: Override /std:c++ for MSVC (930b1f0)
• Make: Add CUDA to GitHub CI (2a4552c)
• Make: Workaround CI issues (46410c6)
• Make: No bare builds on Windows & macOS (b42a340)
• Fix: MSVC compilation issues (5f60fe4)
• Improve: Simplify setting thread-counts (20bbe22)
• Make: Install sz before szs in CI (9033649)
• Make: Skip x86 intrinsics in universal builds (ce6cab2)
• Fix: Inferring Ice Lake similarity kernels (a65dc99)
• Fix: Disambiguate szs_ symbols (1fd5db8)
• Make: Override VS Code compiler choice on osx (d06d4e4)
• Make: Avoid SVE builds on macOS (abb970b)
• Fix: Wrong SZ_DYNAMIC_DISPATCH check (acdab3f)
• Make: Preinstall wheel in CI (62f8c98)
• Make: Bump tapes to 4.0 (72fd6be)
• Fix: Avoid forced inlining for HW flags (4cef520)
• Fix: Feature-checking STL (b1418b5)
• Fix: Missing allocator_traits include (44b8d72)
• Fix: Workaround for static_cast (b6a4cc6)
• Fix: Avoid unaligned XMM loads (0f840bb)
• Fix: static_cast to standard for MSVC (43c953f)
• Make: Avoid uv in GitHub CI (89ed74d)
• Fix: Unused variable in group_by (3761107)
• Fix: Unused qsort on MacOS (cd263ae)
• Improve: Unaligned loads in serial hashes (5e9d488)
• Make: Parallel backends CI (352b48d)
• Make: Referencing old tests (7c4fe32)
• Make: Caps introspection flags on Arm (d6c7cf3)
• Fix: Converting to string views (6b00ab9)
• Fix: Unused symbols (e825ed8)
• Fix: Fetching engines ::capability_k (ddc640b)
• Fix: uninitialized intersection count (ef3ca96)
• Make: Disable NUMA by default (b27552d)
• Fix: Guard SVE checks for cross-compilation (dac3941)
• Make: Install Git on Alpine (1ff0c15)
• Make: Log Alpine version (7ea327e)
• Fix: Type-casting seed on Clang (4598f42)
• Make: Bump Fork Union to 2.2.2 (e674413)
• Fix: Deprecate levenshteinDistance in Swift (70c4add)
• Fix: Passing StringZillas doctests (529fb76)
• Fix: Allow NULL allocator args (b0c33bd)
• Make: NVCC flags for Rust (6b38ea2)
• Fix: Report requesting 1 CPU core (4ef0464)
• Improve: Passing StringZillas.rs tests (34bb89a)
• Improve: Use StringTape for GPU backends (87a0767)
• Docs: StringZillas C API (34f4137)
• Fix: Memalloc initialization on MSVC (#230) (a6e0a77)
• Docs: Drop OpenMP and old name (7f45118)
• Improve: Infer capabilities from DeviceScope (a807eba)
• Improve: Introspect sz_device_scope_t (d9100a3)
• Make: Consistent -O2 optimization (5fcde22)
• Improve: Drop unused info1 (f4d4a76)
• Fix: Rendering byte strings in Python (cf73d79)
• Fix: Forward errors from sz_rune_parse (474dec4)
• Improve: PyTest different MinHash dimensions (b5060a8)
• Fix: Expose value_type for CUDA fingerprinter (7724886)
• Fix: Fingerprinting memory management (f8dea13)
• Fix: to_span compilation (e7fdd98)
• Improve: Wrap high-dim fingerprints (bbf30d2)
• Fix: Handling empty strings in arrays (517757c)
• Improve: More readable PyTest (a6d3ed2)
• Fix: Checking for Ice Lake caps (4f7649a)
• Improve: Simpler & slower Py args parsing (36745f4)
• Improve: Propagate error message to Py (711bd63)
• Improve: Comparing 2 mem-allocators (7870cd3)
• Fix: Skip missing affine_levenshtein_utf8_ice_t (1e112c6)
• Make: Custom CudaBuildExtension for Python (4abf63c)
• Make: Option to disable CUDA builds (84492e5)
• Fix: Refer to prong_t in executor concepts (57d4ec8)
• Fix: MSVC & Clang compilation errors (31fcdb3)
• Make: Avoid OpenMP in builds (183ea96)
• Fix: Announce LevenshteinDistancesUTF8Type (cb81c00)
• Improve: Cache hardware capabilities (b9a1109)
• Improve: Export capabilities as a tuple (0223b62)
• Improve: Printing CUDA caps (94d8c36)
• Fix: Match Apache Arrow layout (38c87b4)
• Improve: Type-casting seeds in Strs.sample (5ac53c3)
• Improve: Constructing Strs from PyArrow (7aebef4)
• Fix: Track ownership of Strs offsets (65380ca)
• Improve: Expose sz_capabilities in non-dynamic builds (6206eb4)
• Fix: Avoid depending SZS -> SZ (c411e37)
• Docs: Mark programming languages correctly (d1fc68c)
• Make: Bump C++ & CUDA to 20 for libs (320f3da)
• Fix: rebind_alloc in C++20 (492d726)
• Fix: Tautological compare check (b924d90)
• Improve: Random-access similarity outputs (c5e778d)
• Make: Building parallel Python packages (0f44c25)
• Fix: Clang build warnings (7741272)
• Fix: Using braces for Clang builds (325cedb)
• Make: Pull submodules in CI (5bb90b3)
• Fix: Avoid _mm256_cvtepi64_epi32 on Haswell (014002e)
• Make: Separate parallel library sources (ff019b1)
• Make: Forward march flags through NVCC (496ae84)
• Make: Move CUDA lib into header (d4a66c5)
• Make: FMA flag for Haswell (2e1daa4)
• Fix: Compilation of all C targets (a2b228c)
• Make: Compiling StringZillas shared libs (6ac80e8)
• Improve: gpu_specs_fetch & GPU args order (3d7b491)
• Docs: Sync description one-liner (ab9b617)
• Improve: Draft parallel fingerprinting API (e49f570)
• Improve: Runtime variable window widths (031d067)
• Make: Rename lib.rs (5d82454)
• Docs: Reuse operators state (e5e2702)
• Improve: Naming multi-input processors (a3c3510)
• Fix: Scramble results between fingerprint benchmarks (fbf7203)
• Make: Format Python to 120 columns (640b7c4)
• Improve: Naming internal symbols (3b48c93)
• Improve: Unroll CUDA fingerprints (37f3d80)
• Docs: Refresh Python benchmarking suite (49cf4ea)
• Fix: Weird compiler bug related to cuda_status_t (0a0955e)
• Fix: Fingerprinting in CUDA (52b1d73)
• Fix: Estimating hash counts in fingerprints (058af71)
• Improve: Unroll & parallelize fingerprinting (cda36fd)
• Fix: Inferring the prong type of executors (531b1e9)
• Improve: Align thread-pool within stack-frame (b1077a4)
• Improve: Wording inconsistencies (afaf11b)
• Make: Launchers for Parallel C++ benchmarks (9f3beac)
• Fix: Fingerprinting via Skylake extensions (08c1e86)
• Fix: Passing fingerprinting builds (44a058d)
• Improve: Include hash counts in fingerprints (0d9ba5b)
• Improve: Consistent kernel naming without underscore prefixes (05725b2)
• Improve: Expose floating-point SIMD states (4a57789)
• Fix: Consistent barrett_mod in C++ & Python (1fc1cab)
• Docs: Using uv for tests (f86dfaa)
• Fix: Choosing co-primes with std::gcd (a1b3001)
• Improve: Using fast calling convention for CPython (97ab23c)
• Docs: Show higher recall with better hashes (255d443)
• Improve: Ensure seed affects hashes (78d39f9)
• Improve: Separate StringZillas Python code (883a3cd)
• Fix: Fingerprinting compilation (19f92c4)
• Improve: Explore Min-Hashes (46dd7d0)
• Improve: Test fingerprint equivalence (e4aa3f7)
• Fix: is_same_type usage over std::is_same (0ab5710)
• Improve: Ignore previous UB commit in blame (7fc7323)
• Fix: Avoid UB with underscore prefixes (74e3b6f)
• Fix: sz_bitcast strict aliasing (80b97de)
• Fix: Avoid std::swap in device code (c0aea26)
• Fix: C++17 compatibility issues (7ea685d)
• Fix: Guard C++20 concepts use (60763b3)
• Fix: Backport std::remove_cvref to C++17 (ffee12b)
• Improve: Move safe_vector (78d8c96)
• Fix: Limit constexpr use in C++11 (866e2f2)
• Fix: Minor build issues (46a6d63)
• Improve: Extend dummy executors API (498d72a)
• Fix: Wrong Fork Union class name (5def3af)
• Improve: Compile-time-known span extents (8355b6e)
• Improve: Move arrays_equality (e96d26f)
• Improve: Merge fingerprinting drafts (cf6077e)
• Make: Deprecate Find Many kernels (a6f799f)
• Improve: Extend find_many tests (c80ce60)
• Improve: Upgrade Fork Union (fb5f429)
• Docs: Similar wording in "Explore Levenshtein" (766e250)
• Fix: Correct namespaces for scripts (aafcbbd)
• Fix: Replace +g with +m,r like GB (40bd3ed)
• Fix: Wrong boundary conditions for count_many_parallel (0b22dd9)
• Improve: Switch to "StringParaZilla" naming (0f3f928)
• Improve: Cleaner haystack splitting (3e93b75)
• Improve: Prioritize find-many tests (7a433a9)
• Improve: SZ_DYNAMIC attributes (2f0334a)
• Fix: Forwarding dataset nothrow-copyable views in tests (170b61b)
• Improve: Use CUDA Atomics to aggregate globally (cf7d9a5)
• Make: Missing bench_search target (1133cce)
• Fix: find_many.cuh compilation issues (72affc2)
• Fix: AC dictionary try_assign with different alloc (4f9d2db)
• Fix: Warning around immutable span conversion (d88842a)
• Docs: Target names (503e470)
• Fix: Match C++ class names (966dd09)
• Docs: Sections & links (f5a94b1)
• Make: Bump fork_union (3f5de03)
• Fix: Revert to separate find_many algos for different length (0acbeeb)
• Improve: __reduce_max_sync in SW on Hopper (c45017d)
• Fix: unified_alloc propagation (13e0201)
• Docs: Move segmenting & features drafts (3972cb3)
• Fix: Compiler over-optimizing bench_find_many (16f5fa4)
• Fix: Prefix length in parallel counting of short needles (69c7b6a)
• Fix: Including the entire haystack into match (3ee1676)
• Fix: Buffer overflow due to wrong thread count (887c16b)
• Improve: Skip vocabulary duplicates (5302a4d)
• Improve: New multi-pattern search APIs (20c9135)
• Fix: Missing std::generate include (853a0fa)
• Improve: Multi-byte characters support (85c5bf8)
• Improve: Propagate allocators in safe_vector (0c442d1)
• Improve: Custom validators for nullary benchmarks (0cf994a)
• Improve: Benchmark early exit (aaa6927)
• Make: Rename bench_search -> bench_find (6f65624)
• Docs: Aho-Corasick CUDA design (496e55a)
• Improve: New caching primitives (df1f7ef)
• Fix: Checking STRINGWARS_STRESS env-var (ade74f6)
• Improve: Support executors in multi-pattern search (35fad3d)
• Improve: Divergent branches on i16 SW on Hopper (817b15c)
• Fix: OOB impact on SW scoring (cd9ff1e)
• Fix: NW & SW on Hopper (4134e44)
• Improve: More noticeable signaling in tests (ece416d)
• Improve: Scheduling speculative kernels (8a6a185)
• Improve: Run multiple warps per block (3b2f263)
• Fix: Comparing Affine benchmark results (64d8f4d)
• Improve: Fuzzy test Ice Lake kernels (d603f7d)
• Improve: Shrink Affine Ice Lake kernels (79e7a2f)
• Fix: Affine top row initialization (b9e4160)
• Fix: Levenshtein w. Affine costs on GPU for zero-length ins (076e58a)
• Improve: Test weird affine gaps (3bbebc8)
• Improve: Match fork_union API in executors (1d6f58d)
• Improve: Use fork_union pools (81463d4)
• Docs: Inconsistent naming (8b62f2c)
• Make: Add fork_union dependency (bd3d341)
• Improve: Duplicate bytesum assignment (bfcd10e)
• Make: Bump CUDA version (c709621)
• Fix: Ice Lake calls for empty inputs (f59ba5e)
• Fix: Correcting blends on Kepler (563a73d)
• Improve: Scheduling Levenshtein in CUDA (77b1087)
• Improve: Avoid views::group_by with callback (6a61e1b)
• Improve: bytes_per_cell_t enum (c6e907a)
• Fix: Inconsistent timing in bench_unary (60b99ab)
• Improve: Bounded methods not supported (23a7f58)
• Improve: Use requires clause (22c691e)
• Improve: Naming "executor" interfaces (7e778d9)
• Improve: Generalize Levenshtein in CUDA (29da524)
• Fix: Inclusion guard macro names (8e79fb7)
• Docs: More datasets on HuggingFace (0b33ac3)
• Improve: Aligned ZMM diagonal stores (10b5405)
• Improve: Branchless K-mask calculation (3998c1f)
• Improve: Measure gap magnitude (b4eb6a4)
• Fix: Avoid horizontal walker overflow (9b2b4e5)
• Fix: All Gotoh baselines (84397ae)
• Fix: Initializing affine DP matrices (640853f)
• Improve: Differentiate unary & uniform costs (8c15447)
• Fix: Horizontal Affine Walkers (e5d85f3)
• Improve: Alloc type-size check in safe_vector (9c5a56c)
• Improve: Fetch warp-size dynamically (83dd5fb)
• Improve: Warp-shuffle reductions in SW (914e98d)
• Fix: Over-estimating number of overlapping matches (b85d3ca)
• Improve: Faster multi-needle tests (0895d28)
• Improve: Splitting jobs in baseline multi-search (d47d849)
• Fix: Slicing corner-cases in OpenMP (7bcc803)
• Improve: Use std::execution for baseline tests (f228264)
• Improve: Parallel baseline for substring search (6ebc7b0)
• Fix: bytes_per_core_optimal estimate (83bc966)
• Improve: Pointer-constructible spans (a0fd136)
• Fix: Passing multi-needle tests (faad971)
• Fix: Calculating find_many_match_t properties (d0ebee8)
• Fix: Indexing needle IDs (42fa08d)
• Fix: Use smaller types in BFS queue (0c6dd00)
• Fix: Aho Corasick construction (e27f86f)
• Improve: Consistent shuffling behavior in benchmarks (d4d55fa)
• Fix: sz_copy_skylake tail handling on large input (#222) (6da5e1e)
• Fix: Propagate substitutions to benchmarks (eabe605)
• Fix: Underutilized 99% of the H100 (7a9a243)
• Fix: Using OpenMP directives (41e1a6e)
• Fix: Forward GPU specs in CUDA tests (9d86d4c)
• Improve: Generalize memory requirements estimates (7bd92c5)
• Fix: Missing STL includes (f7d365a)
• Improve: Use CUDA constant memory (cebd180)
• Improve: Forward-declare substituters (0a04960)
• Improve: Show signed integers in SIMD types (382c05d)
• Fix: i16 Ice Lake NW/SW alignment (8cc9794)
• Improve: Catch & log exceptions (cb739e2)
• Improve: Better UTF8 tests for similarity (b04e934)
• Make: Parallel test launchers (1ece547)
• Fix: Build issues (9db0287)
• Fix: Included filename (ee38a22)
• Fix: Uniform costs for UTF-32 runes (6edcf7f)
• Improve: New template SFINAE in similarity.hpp (10279f9)
• Fix: Initializing horizontal aligner (91df0ce)
• Improve: Use tagged C enums (35ba76c)
• Improve: Allow custom validators in benchmarks (bd2a21d)
• Fix: Compiling constant_iterator in CUDA (bc59ee3)
• Fix: std::allocator::rebind deprecated (4c87404)
• Fix: Avoid std::iterator dependency (b3db596)
• Make: Move similarity benchmarks (90bfcb6)
• Fix: Report error codes in tests (05f17a6)
• Make: Separate Parallel C++ and CUDA tests (b175103)
• Make: Rename test files (2eeab83)
• Fix: Passing CUDA similarity tests (4c75d81)
• Fix: NW/SW test correspondence (fea39ed)
• Improve: Differentiate min/max-imizers (c96c5ed)
• Improve: Annotate throwing exceptions (2ef667c)
• Fix: Missing sz_i16_t definition (238c86d)
• Fix: Avoid similarity scoring references (e4b1bbd)
• Improve: Shorter type aliases (3ef1d26)
• Make: Revert to C++ for core tests (0c6ff1f)
• Docs: StringCuZilla design choices (f42aa85)
• Make: Separate StringCuZilla (c5fd4bc)
• Improve: Move capabilities to types.h (1c1582f)
• Make: Compile with OpenMP (1671b0f)
• Improve: Allow datasets in VRAM (b1c9a74)
• Fix: Shuffle datasets with over 4B tokens (247c6ec)
• Fix: Overflow mean_token_length calculation (efcadd1)
• Make: NVCC kernel debugging symbols (0c0ff42)
• Fix: Arrow-like string array (fa4b0f4)
• Fix: Overwriting alignment scores (668a386)
• Fix: Synchronizing CUDA kernel launch (3b85a00)
• Fix: Shared memory requirements (27ad8f5)
• Make: NVCC can't handle fsanitize (ea7647f)
• Make: Draft CUDA compilation (1b96ef4)
• Fix: Hardening malloc(0) behavior (7524882)
• Improve: Share C++ macros and typedefs (e82d045)
• Fix: Accounting for different gap costs (e4e517f)
• Fix: NVCC warning for negative size field (a6c0fa2)
• Fix: Track capacity in fixed buffer alloc (64b40a9)
• Fix: Sign-cast warning in _mm256_set1_epi64x (aac2e8f)
• Fix: Overriding SZ_DEBUG macro (bca734a)
• Fix: Calling unused helper struct unit tests (d369170)
• Improve: Cleaner API for OpenMP (bc311b3)
• Fix: Shifting Levenshtein diagonals in OpenMP (6b5ef98)
• Fix: Unaligned loads/stores of hash state (37863c9)
• Improve: Expose rune-parsing headers (2727a87)
• Fix: Diagonals depth (ef53f75)
• Docs: Showcase indexing diagonals (b55c696)
• Fix: sz::lookup examples in Rs (778d4f0)
• Fix: Compiling SVE on MacOS (42270c8)
• Improve: Inline cheap calls (28282d2)
• Make: List scripts/ deps for uv (1907d2b)
• Docs: Formatting (dd57536)
• Docs: uv instructions (9460fd4)
• Fix: Unaligned sz_hash_state_t stores (7e65a1e)
• Improve: Align inner hash-states (1b3cdd5)
• Improve: Use GiB over GB (811fc59)
• Improve: Construct Byteset::from_bytes (34660f2)
• Fix: Remove missing Ice-Lake benchs (0c08564)
• Fix: Compiling Py bindings (7e08180)
• Make: CMake formatting (44485fb)
• Docs: Formatting and references (928fd79)
• Fix: No return (4cb096b)
• Docs: Listing bench details (343b858)
• Make: Patch passing SZ_USE_SVE definitions (45b15b0)
• Fix: Expanding feature-detecting macros (75ef77e)
• Improve: Bold benchmark names in CLI (2ab635e)
• Improve: fill_random checksums in benchmarks (0bba772)
• Improve: Simpler SVE find nested loop (a1604b7)
• Improve: Unrolled serial hashing (77e482e)
• Fix: find_sve mask update on long needles (a493ab8)
• Improve: More simple substring search tests (68449fd)
• Fix: bench_sequence CMake target (905749c)
• Fix: Drop double negation in logging (3d77ec6)
• Improve: Use SQINCP in SVE for increments (efda23b)
• Fix: Dispatching SVE kernels (06ea5f7)
• Docs: SVE2 intersects TODO (fafb8b0)
• Improve: do_not_optimize token-level results (d1a3779)
• Make: Rename bench_sort (991a78b)
• Improve: Naming benchmark names (0074ad7)
• Improve: Better sorting benchmarks (7d534fb)
• Fix: Computing improvement percent (a5de795)
• Improve: Faster equality checks on NEON/SVE (20f35c7)
• Improve: New token-level benchmarks (12e1edd)
• Docs: Describe trivial types (af686dd)
• Fix: Naming byteset signature (9676cdb)
• Improve: Naming "vtable" entries (b9794e5)
• Make: Upgrade to C++20 for benchmarks (aa7f275)
• Improve: New-style "container" benchmarks (3f1c723)
• Fix: Reverse order std::search offsets (244e605)
• Docs: Ignore formatting CMake (366816e)
• Make: Formatting CMakeLists.txt (467b4b8)
• Fix: Extra comma in printf (298d214)
• Docs: Outdated function naming & spelling (92b9a56)
• Improve: Token benchmarks (3b1897e)
• Improve: Logging in container benchmarks (4d955d3)
• Fix: No intersect for Skylake (48d70ea)
• Fix: Revert to XMM on Haswell (4bec1e5)
• Fix: Composing STL collections (f9da4ed)
• Fix: std::string::data is mutable only since C++17 (ff23c3d)
• Improve: Discard state in streaming hash (828263f)
• Improve: Discarding buffer in streaming hashes (c4f7a0e)
• Improve: Separate PRNG backends in benchmarks (af54e93)
• Fix: Guard Skylake benchmarks (2965502)
• Fix: Unused _sz_capabilities symbols (8bb90e5)
• Fix: sz_intersect signature (f712de3)
• Make: Don't build stringzilla_bare on MacOS (a7b35ba)
• Fix: Variable in C++14 constexpr (feb415f)
• Fix: Unused Levenshtein tests (90540d3)
• Fix: find_1byte signature compatibility (d19e8b8)
• Improve: Fix minor inconsistencies (f656577)
• Docs: Exploring perfect Unicode hashing (197cd87)
• Improve: Test set intersections (1d95601)
• Fix: Randomization benchmarks (8dc4a2c)
• Docs: Formatting (5c02c4e)
• Docs: Details on the Unicode range (b6e4406)
• Docs: Ignore C++ docstring updates blame (407dd2d)
• Docs: New formatting in C++ (0d982a4)
• Fix: Passing sz_sequence_t::handle (75fabf1)
• Improve: Remove redundant comments from sz_hash_state functions in Rust (9fe25df)
• Improve: Expose sz_lookup (471b002)
• Improve: Expose sz_hash_state_init, sz_hash_state_stream, and sz_hash_state_fold to Rust (1757e4e)
• Improve: Exposed sz_move, sz_fill, and sz_copy for Rust (b2085cc)
• Improve: Inline most common Rust APIs (a30b5b7)
• Make: cibuildwheel env variables (fbf256a)
• Make: Decremental Rust builds (8877c82)
• Fix: Detecting caps in dynamic builds (d52bf63)
• Fix: fill_random test condition (8b396c8)
• Fix: Compilation of all bindings (2caefac)
• Make: Drop unused build.sh (2bbafa1)
• Improve: Testing hash functions (80688bb)
• Fix: Passing new hashing tests (268af53)
• Improve: copy/move on Haswell with interleaving (69dfa10)
• Docs: Announce JOINs (2225488)
• Improve: Ordering includes (6e71536)
• Improve: Vectorize sz_equal_haswell (7aad4bb)
• Docs: Explaining compare.h operations (d7bab8d)
• Improve: Clean memory.h header (7698392)
• Improve: Use default allocator, when not provided (5a12c00)
• Docs: Disable sorting includes (8bc161f)
• Fix: Ice Lake partitioning logic (1da0e2b)
• Improve: Expose Insertion-sort helpers (a38867f)
• Fix: Merge-step bug in stable sort (db61d93)
• Improve: Introduce typed _sz_swap macro (dcf6c65)
• Improve: Rename sz_sort to sz_qsort (6191cc6)
• Fix: sz_sort_serial passes tests (8bad799)
• Fix: uniform_int_distribution lower bound (bdee111)
• Fix: sz_sort_serial passes for same length inputs (0fda5a5)
• Improve: Drop hybrid sort code (50d8291)
• Fix: Underflow in serial sorting (5970fa4)
• Make: Recommend pretty-printing GDB symbols (a818f97)
• Fix: uniform_int_distribution upper bound (17f28a3)
• Fix: In C++11 constexpr constructor must be empty (13bace2)
• Fix: Sorting benchmarks for new API (66f2ac9)
• Improve: Separate fingerprinting benchmarks (187e0bd)
• Make: Renamed temp-git-split-file -> scripts/bench_token.cpp (031bedf)
• Make: Renamed scripts/bench_token.cpp -> temp-git-split-file (07d2239)
• Make: Renamed scripts/bench_token.cpp -> scripts/bench_fingerprint.cpp (a0318eb)
• Docs: Signatures and typos (982dd4d)
• Improve: Wrap std::accumulate for checksums (bce107a)
• Improve: Validate checksums in benchmark (abe8d07)
• Fix: Tail sum order in checksum_haswell (b20d7cd)
• Fix: Infer allocators value_type (5bbd971)
• Fix: Tail handling in sz_checksum_haswell (84cb4c8)
• Fix: Loops in AVX-512 checksums (4044855)
• Fix: Loop in sz_checksum_haswell (509b58b)
• Improve: Relax many constexprs from C++20 to C++14 (0a3e363)
• Make: Move drafts (1de3166)
• Make: Renamed temp-git-split-file -> include/stringzilla/hash.h (5a36cb7)
• Make: Renamed include/stringzilla/hash.h -> temp-git-split-file (7052266)
• Make: Renamed include/stringzilla/hash.h -> include/stringzilla/fingerprint.h (0ef7cf1)
• Docs: Spelling usnigned (d18a159)
• Improve: hybrid bench sort performance (9880f26)
• Fix: hybrid bench sorts loading initial stirng bytes incorrectly (455508f)
• Fix: stable sort bench tests failing (821d19e)
• Fix: Minor dispatch issues (d20e589)
• Improve: Faster levenshtein_baseline (d9557d3)
• Fix: BMI flags for BZHI (fa47deb)
• Fix: Masks back to using BZHI (bd7054e)
• Make: Library namespaced aliases (f3811d7)
• Fix: sz_u512_vec_t members visibility (2007d49)
• Fix: Bounded Levenshtein returns (749b0d8)
• Fix: Skylake dispatch (48e0913)
• Fix: Linking stderr (084d653)
• Docs: Formatting docstring (c99daf3)
• Fix: Initializing basic_charset (864ee03)
• Fix: Correct basic_charset operator (#203) (e20d207)
• Improve: Ignore 40 commits in blame (064829a)
• Fix: Overriding LibC in 32-bit Windows (645539b)
• Improve: C++ version macros naming (19c2ae9)
• Make: Detect Apple Universal builds (6d61c21)
• Make: Rename stringzillite to stringzilla_bare (364e2ca)
• Fix: Symbols names & visibility (406bf0f)
• Fix: Haswell compilation flag (00f27f6)
• Fix: Filter compare.h file (6512f1d)
• Make: Split ./include/stringzilla/find.h to ./include/stringzilla/compare.h (fc9e5d6)
• Make: Split ./include/stringzilla/find.h to ./include/stringzilla/compare.h (49e8d9d)
• Make: Split ./include/stringzilla/find.h to ./include/stringzilla/compare.h (fc408fa)
• Fix: Partially filter stringzilla.h file (41e5917)
• Fix: Minor macro mismatches (5f7ca59)
• Fix: Filter types.h file (b835051)
• Fix: Filter sort.h file (1ba7982)
• Fix: Filter small_string.h file (5b55e19)
• Make: Separate builds for Skylake & Ice (4a1f03c)
• Improve: Platform-specific equality checks (8b44d6a)
• Fix: Filter hash.h file (be4c63d)
• Fix: Filter similarity.h file (8b401bd)
• Fix: Filter memory.h file (295d49a)
• Fix: Filter find.h file (2a1fcd1)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/memory.h (2f76521)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/memory.h (45e57ee)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/memory.h (66778d6)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/sort.h (c357c3e)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/sort.h (cbfe5c7)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/sort.h (085d2d3)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/small_string.h (3464cb4)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/small_string.h (89c4681)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/small_string.h (3f9c248)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/similarity.h (e23c35f)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/similarity.h (10d829e)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/similarity.h (d74e5dc)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/hash.h (1f60e6d)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/hash.h (08d0a20)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/hash.h (9e9f256)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/find.h (974ed78)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/find.h (14ba3bf)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/find.h (9e577be)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/types.h (8cb0742)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/types.h (22e3d1e)
• Make: Split ./include/stringzilla/stringzilla.h to ./include/stringzilla/types.h (ecb3775)
• Fix: Wrong env. variable names (d0678f8)
• Make: Inline ASM for detecting CPU features on ARM (0ee549a)
• Fix: Default Levenshtein upper bound (62ca6a0)
• Improve: Levenshtein functions for unicode (d3b423a)
• Docs: Levenshtein tutorial in Jupyter (715ad10)
• Fix: sz_look_up_transform_avx512 declaration (585f7d5)
• Improve: #pragma region dashes (fe4449b)

Release: v3.12.6 [skip ci]

Patch

• Fix: Skip second half of AVX2 copy (a198630)
• Fix: LUT API name (c78adca)

Release: v3.12.5 [skip ci]

Patch

• Make: Upgrade to newer CMake (25224bb)

Release: v3.12.4 [skip ci]

Patch

• Fix: Long input tail in sz_copy_avx512 (#221) (18f04e7)

Release: v3.12.3 [skip ci]

Patch

• Improve: C++ Lifetime bounds (caf5650)

Release: v3.12.2 [skip ci]

Patch

• Fix: Horspool corner-case (#216) (ffa4518)

Release: v3.12.1 [skip ci]

Patch

• Fix: Uninitialized bad shift table (#213) (d2480a5)

Together with @MarkReedZ we've added basic GoLang bindings to StringZilla, which look surprisingly fast compared to native GoLang strings. We currently use the new cGo annotations available in Go 1.24:

Cgo has gained new capabilities in Go 1.24, supporting new C function annotations to improve runtime performance. Among them, #cgo noescape cFunctionName is used to inform the compiler that the memory passed to cFunctionname will not escape; #cgo nocallback cFunctionName indicates that this C function will not call back any Go functions. In addition, Cgo's inspection of multiple incompatible declarations of C functions has become more stringent. When there are incompatible declarations in different files, errors can be detected and reported more timely and accurately.

I was using an Intel Sapphire Rapids machine on AWS for preliminary testing and benchmarking. I've precompiled StringZilla with dynamic dispatch enabled, linked to the thin GoLang binding layer:

$ ~/StringZilla/golang$ CGO_CFLAGS="-I$(pwd)/../include" \
        CGO_LDFLAGS="-L$(pwd)/../build_golang -lstringzilla_shared" \
        LD_LIBRARY_PATH="$(pwd)/../build_golang:$LD_LIBRARY_PATH" \
        go run ../scripts/bench.go  --input ../leipzig1M.txt --split lines --seed 42

... and compared to native GoLang strings on some key operations:

Benchmarking on `../leipzig1M.txt` with seed 42.
Total input length: 129644797
Total lines: 1000000
Average line length: 128.64
Running benchmark using `testing.Benchmark`.
strings.Contains              :      309           3818144 ns/op
sz.Contains                   :      664           1881251 ns/op
strings.Index                 :      325           3669081 ns/op
sz.Index                      :      624           1990093 ns/op
strings.LastIndex             :       12          85201713 ns/op
sz.LastIndex                  :      494           2306318 ns/op
strings.IndexAny              :  6321228             181.0 ns/op
sz.IndexAny                   : 10608960             112.6 ns/op
strings.Count                 :      156           8015292 ns/op
sz.Count (non-overlap)        :      285           4206698 ns/op
sz.Count (overlap)            :      284           4204370 ns/op

So if you are processing a lot of text in Go, try doing so with StringZilla and stay tuned for the upcoming 4.0 release #201 🥳

Release: v3.11.3 [skip ci]

Patch

• Improve: Pointer casting rules (a3f2f00)
• Docs: LLVM build instruction (89be0cb)

Release: v3.11.2 [skip ci]

Patch

• Make: Using SIMD on FreeBSD (#205) (6700fcc)

Release: v3.11.1 [skip ci]

Patch

• Fix: Matching N3322 for memcpy UB in C2y (14ee92c)

• 🆕 sz_checksum(char const *, size_t) C 99 interface
• 🆕 sz::str().checksum() C++ 11 interface
• 🆕 sz.checksum(str) Python interface

Database and other Systems Engineers, you can now use StringZilla to dynamically dispatch different check-sum kernels for AVX2 capable Haswell+ CPUs, AVX-512BW capable Ice Lake+ CPUs, and Arm NEON CPUs on mobile. In AVX-512, masked loads are used extensively, resulting in a 10% improvement even on typical English words, averaging 5 bytes in length and 20x performance improvement compared to the serial code for longer strings.

On the technical side, on x86, the kernels use the well-known SAD(text, zeros) idiom to accumulate absolute differences between individual bytes into 64-bit words. It also uses bidirectional traversal to saturate the core, capable of performing 2 loads per CPU cycle. Moreover, on large inputs, it switches to streaming loads, separately handling the head and the tail, similar to our memcpy alternative, also outperforming LibC on AVX-512-capable machines 😎

Minor

• Add: Checksums in Python (1b77de9)
• Add: Checksum tests (c2b997c)
• Add: Checksum kernels (a99337b)

Patch

• Docs: Simpler Python doc-strings (ad5fa2c)
• Fix: sz_checksum visibility (9bec0eb)
• Fix: Missing _mm_cvtsi128_si64x in Clang (c8c6c7c)

Release: v3.10.11 [skip ci]

Patch

• Make: Pin cibuildwheel==2.21.3 (592034c)
• Make: Bump TinySemVer CI (358f4ad)
• Fix: Missing C++ function-objects (4be9409)
• Fix: Equality check on NEON (#197) (705fb5e)
• Improve: Longer equality check (2af873a)