Skip to content

perf: optimize seeded XXH3 and streaming XXH32 - #16

Merged
tisonkun merged 1 commit into
mainfrom
codex/optimize-seeded-xxh3-xxh32
Sep 2, 2026
Merged

perf: optimize seeded XXH3 and streaming XXH32#16
tisonkun merged 1 commit into
mainfrom
codex/optimize-seeded-xxh3-xxh32

Conversation

@tisonkun

@tisonkun tisonkun commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • derive the seeded XXH3 default secret through a word-oriented runtime path while preserving the index-based const path used by const constructors
  • arrange XXH32 streaming accumulators so AArch64 code generation moves input multiplication out of the accumulator dependency chain
  • preserve the public API, digest compatibility, no_std support, and allocation-free implementation

Design Notes

The seeded XXH3 runtime and const derivation paths intentionally use different iteration forms. This avoids private helper calls on affected Apple LLVM code generation without forcing inline expansion that regresses Linux AArch64; a unit test verifies both paths produce identical secrets.

XXH32 stores adjacent logical accumulators in reverse order internally. This allows AArch64 LLVM to use a vector load, multiply, pair reversal, and vector accumulator update while restoring the canonical logical order for finalization.

Benchmarks

Apple M4 Max, AArch64 macOS, rustc nightly 1.99 / LLVM 23:

  • seeded XXH3-64, 241 B: 42.09 ns -> 5.12 ns (twox-hash: 6.54 ns)
  • seeded XXH3-64, 4 KiB: 116.9 ns -> 83.32 ns (twox-hash: 83.98 ns)
  • XXH32 streaming, 1 MiB in 64 KiB chunks: about 6.45 GB/s -> 6.92 GB/s (twox-hash: 7.20 GB/s)

Native AArch64 Linux container, rustc 1.90 / LLVM 20:

  • seeded XXH3-64, 241 B: 5.61 ns (twox-hash: 6.91 ns)
  • seeded XXH3-64, 4 KiB: 85.03 ns (twox-hash: 85.03 ns)
  • XXH32 streaming, 4 KiB in 64 B chunks: 12.10 GB/s (twox-hash: 12.29 GB/s)
  • XXH32 streaming, 1 MiB in 64 KiB chunks: 12.25 GB/s (twox-hash: 12.26 GB/s)

Validation

  • cargo x test
  • cargo x lint
  • cargo x build --locked
  • cargo x bench
  • focused seeded XXH3-64, seeded XXH3-128, and streaming XXH32 benchmarks on macOS and native AArch64 Linux

@tisonkun
tisonkun merged commit 662c5a2 into main Sep 2, 2026
9 checks passed
@tisonkun
tisonkun deleted the codex/optimize-seeded-xxh3-xxh32 branch September 2, 2026 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant