2026-07-14 · the served path: HTTP + encoder + release build, on a real 115k-chunk workspace

Setup: the number every earlier entry flagged as owed: "a served deployment adds HTTP, encoder, and concurrency." Measured end-to-end over POST /v1/recall against the running release-build server on a real 115,643-chunk workspace (live Gmail + Drive, real ACL cardinality: 762 principals), owner scope (sees ~all chunks, the broad and expensive case), single-threaded warm client. Apple M3 Pro, ParadeDB pg17 in Docker.

QPS under load (closed loop, N=4/16/64): end-to-end HTTP recall saturates at ~50 QPS; the in-process serving core (100k synthetic) saturates at ~120 QPS. The encoder's single ONNX session is the served ceiling; Postgres is the core ceiling.

• Release ≈ dev for recall, because recall is Postgres-bound, not Rust-bound. The heavy lifting is pgvector ANN + pg_search BM25 inside Postgres; a release compile barely moves it. The lever is Postgres, not the binary.
• The get path holds its claim end-to-end: 1.4ms p95 over HTTP, comfortably under the <5ms budget. This is the correct inner-loop hot path.
• Latency is flat across a 112× selectivity change (broad ~115k vs narrow ~1k visible, both ~70ms p50): filtered-ANN + ACL cardinality is not the bottleneck. That retires the ACORN risk on real data. The cost is the encoder (~12–23ms) plus the Postgres hybrid query; the HTTP hop is small.
• The served path exceeds the 50ms envelope: the in-process encode+ANN number (~37–48ms p95) holds, but a real HTTP request adds the served round-trip to ~105ms p95 on this laptop stack. Honest reading: ~20ms p95 scoped hybrid recall in the serving core; ~105ms end-to-end served on a laptop; sub-ms permissioned get; QPS saturates ~50–120 on a single untuned Postgres container. The ≥300 QPS / server-hardware numbers remain unproven, gated on Postgres tuning and encoder pooling, not the Rust build.

2026-07-10 · entity-bound BM25 breach fixed: 542.7ms → 12.6ms p50 (43x)

Fix (migration 0008 + adapter): entity_tags and kind join the bm25 index with keyword tokenizers (the default tokenizer splits account:0, so term_set could never match raw values; the first attempt returned 0 hits), and entity-bound sparse queries become two stages: a Tantivy boolean pre-filter bounding the candidate set to the entity's own chunks, then the exact <@ subset residual over the materialized candidates. Filter-then-rank, never truncate-then-authorize.

• All retrieval paths back inside the <50ms p95 envelope at 1M.

2026-07-09 (later) · QPS under load, and the entity-bound BM25 breach

Setup: 1,000,000 chunks, same machine/profile. All numbers are in-process adapter calls: no HTTP hop, no query encoder (dense/hybrid end-to-end adds ~12ms p50 of client CPU). One shared adapter, 16-connection pool. New latency case: broad principal token AND entity_scope=["account:0"], where broad visibility maximizes the pushed-down candidate set the heap-side entity_tags <@ filter must chew through.

Load (load --sweep --duration-secs 20): closed loop, zero think time; N tasks looping a mixed workload: 70% hybrid recall (1%-selectivity token), 20% current_fact point reads, 10% activity-timeline reads. Latencies include waiting for one of the pool's 16 connections.

• This box saturates at ~165–170 QPS for this mix, by concurrency 4. 16x-ing offered concurrency buys zero throughput, only queue depth. The honest operating point is N=4: hybrid 48.9ms p95 excluding the encoder. With ~12ms encode added, hybrid under load is outside the 50ms envelope even at low concurrency.
• SPEC §4d targets are not met on this setup: target ≥300 QPS hybrid at p95 <50ms per 8-vCPU node; measured ~117 hybrid ops/s inside a 166-QPS mix, on a laptop running Postgres under a Docker VM. The ~2.5x gap is real; measuring on the reference shape is the required next pass.
• Point reads queue behind recall under load: current_fact p95 is 5.4ms at N=4 (vs 0.43ms idle). The get target (≥5k QPS at p95 <5ms) cannot be met from an undifferentiated pool; the planned L3 in-memory projection is the designed fix.
• Entity-bound + broad-visibility BM25 is the worst number ever recorded here: 724ms p95, ~14x the envelope. Dense recall is immune (the router sends tiny entity-bound subsets to exact scan).
• Honesty notes: the bench tenant's activity timeline holds ~2 rows, which is path coverage, not a scale claim. No HTTP hop, no rate limiting, no encoder anywhere in the load loop; a served deployment adds all three.

2026-07-09 (later) · BM25 pushdown: all paths inside the envelope

Root cause (pg_search 0.24.1): the array-overlap operator && is not pushable into Tantivy, and valid_to wasn't indexed, so BM25 queries heap-fetched the entire text match set (~540k rows for a 2-term OR query) to evaluate scope filters before top-k. Fix (migration 0004 + adapter): visibility expressed as a Tantivy term_set (exact overlap semantics inside the boolean; matches nothing on an empty principal set, fail-closed preserved), and valid_to added to the bm25 index. Zero heap-filter nodes remain (~1.9k buffers/query vs ~480k).

• Scoreboard at 1M, worst case per path (p95): get 0.5ms · dense 25.4ms · BM25 17.8ms · hybrid 17.9ms · +12ms encode ⇒ every retrieval path inside the <50ms p95 envelope, encoder included.
• Caveats recorded: term_set shifts raw scores by a constant (+1.0), ranking identical and RRF unaffected; entity_tags <@ subset filtering remains a heap filter, so an entity-bound + broad-visibility case must be benched before claiming that combination; the migration briefly drops search availability during rebuild (~1.6s at 1M).

2026-07-09 (later) · 1M chunks: the valley collapses, then the selectivity router kills it

Setup: 1,000,000 chunks, same machine/profile. Bulk-loaded with secondary indexes dropped (292s; the incremental path was ~12x slower). Along the way: Docker's 64MB default shm silently fails parallel HNSW builds (compose now sets shm_size: 1gb), and an accidental run against the index-less table showed brute-force at 11ms p50 beating HNSW's 72.6ms on the same 1%-filtered query. The planner picks graph traversal exactly where it loses. Fix: an adapter-side selectivity router. A sub-millisecond EXPLAIN row estimate routes ≤20k estimated matches to exact top-k (perfect recall, no graph), the rest to HNSW iterative scan.

• The <50ms p95 claim holds at 1M for scoped dense recall, encoder included: worst dense case 25.4ms p95 + ~12ms encode ≈ ~37ms p95 end-to-end. The valley is eliminated by routing, not tuning.
• The get path is flat with corpus size: 0.29ms p50 at 1M vs 0.33ms at 100k.
• BM25 at 1M is the new breach: ~280ms p50. The fast-field fix that held at 100k does not carry: pg_search scores the full match set before the filters bite, and hybrid inherits it. Until fixed, the honest hybrid number at 1M is ~330ms p95.
• Planner-estimate routing depends on healthy pg_stats; ANALYZE after bulk loads is mandatory.

2026-07-09 (later) · local query encoder measured

Setup: all-MiniLM-L6-v2 ONNX (384-d, matching the chunk schema), CPU-only via ONNX Runtime, single thread, 100 short queries, Apple M3 Pro. Semantic ordering verified by test (related queries must beat unrelated by >0.2 cosine).

• Inside the 5–15ms local-encoder budget. Warm-cache encoder load ~190ms; first-run model download ~4.6s.
• End-to-end dense recall including encoding at 100k / 1% selectivity: ~37ms p50 / ~48ms p95 (additive worst case). That is the number the <50ms p95 claim must hold against as corpus size grows.

2026-07-09 (later) · BM25 fast-field fix

Change: scope-filter columns (tenant_id, visibility, confidentiality) added to the bm25 index as fast fields (migration 0003), letting pg_search filter inside Tantivy instead of on the heap. Same corpus/machine as above.

• 2.7x p99 improvement: BM25 back inside the 50ms envelope at p95/p99. Hybrid tail now dominated by the dense side's 1%-selectivity valley.

2026-07-09 · first curve (Milestone A, week 1)

Setup: 100,000 chunks (384-d vectors, HNSW m=16 / ef_construction=64), Postgres profile (ParadeDB pg17: pgvector iterative scans + pg_search BM25) in Docker on Apple M3 Pro / 36 GB. k=10, 200 queries per case, in-process client: no network hop, no query embedding (local ONNX encoder not yet built).

• The get path already beats its claim: sub-millisecond L1 point reads straight from Postgres, wide margin under the ~2–5ms budget.
• The filtered-ANN valley is real and now located. Latency is non-monotonic in selectivity: 0.1% is fastest (exact scan over a tiny posting set), 50% is near-unfiltered, and ~1% is the worst case: too selective for cheap graph traversal, too large for exact scan.
• All dense paths meet <50ms p95 at this corpus size, but 100k is 1–2 orders of magnitude below the honest ceiling. The next entry must be 1M+ before any latency claim leaves the repo.
• BM25's 87ms p99 is the first SLO breach: pg_search filtering on the heap after the Tantivy match.
• Missing from these numbers: query embedding, network hop, concurrency. Single-query latency only.