docs: P2 — bounded in-sift queue; digest + chunk-embedding caches pulled forward

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

docs/P2.md

@@ -13,8 +13,12 @@ requests** — the deriver stalled for ~20 min until llama-server was restarted.
 never add to that failure mode:
 
 - **Bounded in-flight to Dixie**: at most `dixie.max_inflight` (default **2**) digest/embed
-  calls from sift at once; a request that can't get a permit within 5 s **falls back to
-  `clean`** (with a warning) instead of queueing on Dixie.
+  calls from sift at once. Waiting requests sit in a **bounded FIFO queue inside sift**
+  (`dixie.queue_depth` default **8**, `dixie.queue_wait` default **20 s**); a request that
+  can't get a permit within the wait, or arrives to a full queue, **falls back to `clean`**
+  (with a warning). Sift's queue is visible and cancellable; Dixie's own queue never is —
+  never let requests queue *on Dixie*. (20 s wait + ~15 s digest fits Hermes's 60 s
+  `web_extract` budget.)
 - **Streaming requests** (`stream: true`) for chat completions, and cancel the context on
   timeout, so closing the connection actually stops generation (verify this with a test
   against a real llama-server: start a long digest, cancel at 2 s, confirm the slot frees —
@@ -69,6 +73,28 @@ Out: `search` (P3), cache (P4), knowledge graph.
 6. Fallbacks (always 200 with `warnings`): permit timeout, Dixie error, 45 s timeout,
    empty output → `clean` truncated to the output cap, `digest.fallback: "<reason>"`.
 
+## 2a. Caches (pulled forward from P4, Kyle 2026-09-22)
+
+Content-addressed, SQLite in `/var/lib/sift/cache.db` (`modernc.org/sqlite`), so there is no
+staleness problem: a changed page has a different hash.
+
+1. **Digest cache** — key `sha256(content_hash | normalised_question | model | prompt_version)`
+   → digest text + `chunks_used`. Normalise the question: trim, collapse whitespace,
+   lowercase. Hit → no Dixie call at all (`digest.cached: true`). Bump `prompt_version` whenever
+   the prompt or chunking changes.
+2. **Chunk-embedding cache** — key `(content_hash, chunk_index, embed_model)` → vector
+   (float32 blob). A second question on the same long page skips re-embedding. Also cache
+   the question embedding by normalised question (short TTL not needed; it's content-addressed).
+
+Rules: **Honcho context is never cached** (per-workspace, changes constantly) — fetch it
+fresh and attach to a cached digest; nothing in the cache is workspace-specific. Size-bound
+the DB (default 500 MB, LRU eviction by last-hit time; evict in a background tick). Cache
+failures are warnings, never errors. `/metrics`: hit/miss per cache, evictions, DB size.
+The URL → cleaned-content **fetch cache stays in P4** (it needs TTL/staleness rules).
+
+Tests: hit/miss on identical vs changed content; normalisation; prompt_version bump
+invalidates; context not stored; eviction under size cap; corrupt DB → warn and bypass.
+
 ## 3. Context (`internal/context`, Honcho retrieval only)
 
 - `POST {honcho}/v3/workspaces/{ws}/search` `{"query": q, "limit": 5}`; response is a JSON
@@ -126,8 +152,9 @@ digest_ms fallback`. Never log page content, questions longer than 80 chars, or
 
 - Fake Dixie (`httptest`, streaming SSE) for: prompt framing (document inside tags, system
   prompt present), `enable_thinking:false` + `max_tokens` sent, fallback on timeout / error /
-  empty, permit cap (3 concurrent requests with `max_inflight=2` → one falls back), context
-  cancel propagates (fake server observes the disconnect).
+  empty, permit cap + queue (with `max_inflight=2, queue_depth=1`: 2 run, 1 waits, the 4th
+  falls back immediately; a waiter past `queue_wait` falls back), context cancel propagates
+  (fake server observes the disconnect).
 - Chunk selection: document order restored, token budget respected, no-question path.
 - Context: allowlist, `guardian-star` refused even if configured, 3 s timeout → warning.
 - Recover: Wayback JSON parsing (available/unavailable), `id_` rewrite, fake-success fixtures