P5b and P5c plans: long-document retrieval (BM25 first, background embedding) and /v1/extract + sift extract; given tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/plans/p5b/README.md

@@ -0,0 +1,27 @@
+# P5b implementation plan: questions on long documents
+
+> **For the implementing model:** do not work from this file. The owner gives you one task file
+> at a time. This file is the index for the owner and the reviewer.
+
+**Goal:** P5a refuses `question` on a PDF longer than 200 digest chunks (~240k tokens) with
+"use grep or pages". After P5b such a question is answered: at once from **BM25** over
+page-aware chunks, while the document's chunks are **embedded in the background**; once they
+are, later questions use today's hybrid score. Two facts found while planning:
+
+- The embedding cache (`dcache` `GetEmbed`/`PutEmbed`, `EmbedKey`) exists but **nothing uses it**:
+  every question re-embeds every chunk.
+- `Split` is line-based, so a chunk that starts mid-page carries no `--- page N ---` marker and
+  the model cannot cite its page.
+
+| Task | Package | Given |
+|---|---|---|
+| 01-splitpages-bm25 | `internal/digest` (pure functions) | `digest/splitpages_test.go`, `digest/bm25_test.go` |
+| 02-retrieval | `internal/digest` | `digest/retrieval_test.go` |
+| 03-api | `internal/api`, `README.md`, `skill/SKILL.md` | `api/pdf_retrieval_test.go`; replaces `api/pdf_test.go` (P5a: the 200-chunk default becomes 5000) |
+
+**How this plan was made:** acceptance tests first, no reference implementation; the given tests
+compiled against a panic-only skeleton on master `1311cf3` and failed for the intended reasons.
+
+## Changes during the run
+
+(none yet)