sift/docs/plans/p5b/README.md

P5b implementation plan: questions on long documents

For the implementing model: do not work from this file. The owner gives you one task file at a time. This file is the index for the owner and the reviewer.

Goal: P5a refuses question on a PDF longer than 200 digest chunks (~240k tokens) with "use grep or pages". After P5b such a question is answered: at once from BM25 over page-aware chunks, while the document's chunks are embedded in the background; once they are, later questions use today's hybrid score. Two facts found while planning:

• The embedding cache (dcache GetEmbed/PutEmbed, EmbedKey) exists but nothing uses it: every question re-embeds every chunk.
• Split is line-based, so a chunk that starts mid-page carries no --- page N --- marker and the model cannot cite its page.

Task	Package	Given
01-splitpages-bm25	internal/digest (pure functions)	digest/splitpages_test.go, digest/bm25_test.go
02-retrieval	internal/digest	digest/retrieval_test.go
03-api	internal/api, README.md, skill/SKILL.md	api/pdf_retrieval_test.go; replaces api/pdf_test.go (P5a: the 200-chunk default becomes 5000)

How this plan was made: acceptance tests first, no reference implementation; the given tests compiled against a panic-only skeleton on master 1311cf3 and failed for the intended reasons.

Changes during the run

• Run 1, task 01 (2026-09-26 11:01–12:24): infrastructure, not the model. straylight's llama-server ran out of shared KV (4 slots over one 262,144-token pool). The session got Context size has been exceeded with no commit. Hermes's ornith context was capped at 131072 (Kyle: "b is acceptable"); the task was re-run clean and passed (dca0004, rebased to 7595ad8).
• Task 02, first attempt: my test, not the model or the task. TestEmbeddingsAreCachedAcrossQuestions used longDoc(12), which is 4,617 tokens. That is under SmallDoc (6000), so rule 2 answers it "whole", while the test expected "hybrid". Ornith measured the document and was working it out when I stopped the session (22 min). Fix d52a4fa: longDoc(30) (11,677 tokens, 10 chunks) plus a t.Fatalf precondition. Relaunched from task 02 on a clean tree.