bench

Performance experiments, published with enough detail to rerun them.

First up: what does heavy ingest do to turbopuffer query latency?

turbopuffer: querying while indexing

turbopuffer queues writes into an unindexed backlog and indexes them in the background. The docs say strongly consistent reads can start failing once that backlog passes 2 GiB. I wanted numbers: if you bulk-load vectors into a namespace people are actively querying, what do those queries actually experience?

The benchmark writes random 768-dimensional vectors — no embedding calls, so it isolates turbopuffer write/index/query behavior — while driving concurrent ANN queries. It measures three phases (idle, indexing with backlog under 2 GiB, indexing with backlog over 2 GiB) at both strong and eventual consistency, gating each phase on the namespace's reported index.unindexed_bytes. Writes use disable_backpressure: true to intentionally push past the 2 GiB threshold.

Results

From the canonical run (gcp-us-central1, 2026-06-05, 8 query workers, 60s measured per cell):

Phase	Consistency	Errors	Client p50	Client p95	Client p99	Server p50	Server p95
Idle	strong	0%	53.2ms	65.8ms	79.8ms	16ms	23ms
Idle	eventual	0%	50.9ms	62.5ms	75.7ms	4ms	5ms
Indexing, backlog < 2 GiB	strong	0%	56.4ms	98.5ms	264.8ms	19ms	27ms
Indexing, backlog < 2 GiB	eventual	0%	50.7ms	60.4ms	71.1ms	4ms	5ms
Indexing, backlog > 2 GiB	strong	45%	89.4ms	529.4ms	694.8ms	23ms	32ms
Indexing, backlog > 2 GiB	eventual	0%	72.5ms	440.9ms	626.0ms	4ms	5ms

What I take from this:

• Eventual reads don't notice ingest — until the backlog crosses 2 GiB. Under heavy indexing with the backlog held below 2 GiB, eventual-consistency latency was indistinguishable from idle.
• Strong reads pay a tail tax during ingest. p99 went from 80ms idle to 265ms while indexing, even with the backlog comfortably under the threshold.
• Past 2 GiB of backlog, strong reads get rejected outright. 45% of strong queries returned the documented "too much indexing backlog" error. This is expected behavior when you disable backpressure — the point here is measuring it, not complaining about it.
• Server-reported timings stayed flat the whole time (eventual p95 was 5ms in every phase). The client-side tail growth past 2 GiB happens upstream of query execution. Some of it may be the benchmark's own write traffic sharing HTTP/2 connections, so read the >2 GiB client tails as an upper bound.

Practical version: if you bulk-load into a live namespace, keep the backlog under 2 GiB (leave backpressure on and it's handled for you), and expect strong-consistency reads — not eventual ones — to feel the ingest.

Full output for every run, including namespace metadata snapshots and error breakdowns, is in results/ as Markdown and JSON. Earlier runs in that directory were exploratory (smaller thresholds, backlog that drained mid-phase); the run linked above is the one that cleanly hit all three phases.

turbopuffer: filterable attribute / secondary-index cost

The secondary-index benchmark writes the same 1M random 768-dimensional vectors into fresh namespaces while varying one filterable attribute by type (int or string) and cardinality. It measures ingest throughput, post-write drain behavior, storage/billing signals, and steady-state query latency for ANN-only, ANN+filter, and filter-ordered query shapes.

From the canonical run (gcp-us-central1, 2026-06-07/08, 14M rows total, 100 query phases, 0 write errors, 0 query errors):

Results

Each cell is a fresh 1M-row namespace, fully indexed before queries (2 query workers, 10s measured per cell). The comparable view is ann_filtered at 1% selectivity against the unfiltered ann_only baseline:

Cell	Type	Cardinality	Filtered strong p95	Filtered eventual p95
baseline	none	—	51.8ms (ann_only)	48.1ms (ann_only)
int@100	int	100	61.6ms	62.4ms
int@10k	int	10k	70.5ms	73.1ms
int@100k	int	100k	65.3ms	63.9ms
int@unique	int	unique	69.1ms	67.3ms
string@100	string	100	176.9ms	53.8ms
string@10k	string	10k	66.4ms	67.8ms
string@100k	string	100k	65.0ms	64.5ms
string@unique	string	unique	63.9ms	65.1ms

(int@2/string@2 are omitted: with two distinct values the lowest achievable selectivity is 50%, so they aren't comparable at 1%.)

Sweeping selectivity on int@10k shows the query shape matters more than cardinality — filter_ordered spikes when the filter matches a mid-sized slice that still has to be sorted:

Selectivity	ann_filtered strong p95	filter_ordered strong p95
0.1%	57.3ms	50.5ms
1%	70.5ms	62.2ms
10%	78.9ms	229.9ms
50%	58.5ms	90.2ms
100%	53.0ms	127.4ms

What I take from this:

• No monotonic cardinality tax showed up in ingest throughput. Head/tail baseline drift was +1.2%; most filterable cells stayed inside that band. string@2 was the slowest at ~3.6% below the baseline mean.
• Drain behavior was batchy, not linear. Peak unindexed backlog ranged from 386 MiB to 1.83 GiB; drain duration was a more useful signal than slope for this run.
• Billing stayed flat across cardinality. At 10k cardinality, filterable vs non-filterable added 3.81 MiB for int and 2.86 MiB for string across 1M rows.
• Filtered ANN query latency was mostly modest at 1% selectivity. Strong p95 was usually 61-70ms vs a 52ms baseline; string@100 strong was the notable outlier at 177ms p95 (its eventual reads stayed at 54ms, so the cost is on the strong-consistency path, not the index itself).
• Ordered filter scans had a shape-specific cost. filter_ordered peaked at 10% selectivity (~206-230ms strong p95 across the int sweeps) and recovered at 50% and 100%.

Run the harness directly:

TURBOPUFFER_API_KEY=... TPUF_BENCH_CONFIRM_COST=1 \
  python benchmarks/tpuf_filter_cost_benchmark.py

Use --smoke for a small validation run before the full matrix.

Run it yourself

Cost warning: the default run writes enough data to push a namespace past 2 GiB of unindexed backlog, which costs real money on your turbopuffer account. You must set TPUF_BENCH_CONFIRM_COST=1 to acknowledge this.

cp .env.example .env
# set TURBOPUFFER_API_KEY, TURBOPUFFER_REGION, TPUF_BENCH_CONFIRM_COST=1
docker compose run --rm tpuf-benchmark

Results land in results/ as <namespace>-<timestamp>.md (shareable summary) and .json (full machine-readable output).

For a cheap smoke run that exercises the machinery without the >2 GiB phase:

docker compose run --rm tpuf-benchmark \
  --seed-rows 1000 \
  --max-total-rows 10000 \
  --query-duration-s 5 \
  --query-warmup-s 1 \
  --lt-start-bytes 1MiB \
  --lt-cap-bytes 4MiB \
  --gt-start-bytes 8MiB \
  --allow-small-thresholds

docker compose run --rm tpuf-benchmark --help lists every knob. The ones that matter most:

• --seed-rows: rows written and fully indexed before the idle baseline (default 100000)
• --query-duration-s: measured duration per phase/consistency cell (default 60)
• --lt-start-bytes / --lt-cap-bytes: backlog window for the under-2 GiB phase (default 512MiB–1.75GiB)
• --gt-start-bytes: minimum backlog before the over-2 GiB phase starts (default 2.10GiB)
• --max-total-rows: safety ceiling on total rows written (default 1500000)
• --cleanup: delete the benchmark namespace afterward

Method notes

• Writes go to /v2/namespaces/:namespace with upsert_columns; vectors are base64-encoded little-endian f32 to keep client serialization cheap.
• Queries are random f32 vectors via rank_by: ["vector", "ANN", ...], schema fixed at [768]f32 with ann: true.
• Consistency is set explicitly per query as {"level": "strong"} or {"level": "eventual"} (docs).
• Phase gating polls namespace metadata for index.unindexed_bytes; each result row records the backlog at the start and end of its measurement window, so you can verify the phase stayed in range.
• Each row records request/error counts, success QPS, client p50/p95/p99, and server-side percentiles when turbopuffer returns them.