sift/docs/plans/p5b/03-api.md

P5b task 03: the API answers long-document questions and says how

Branch: p5b (git switch p5b; git status --short must be empty, otherwise stop) Commit subject: api: questions on long PDFs are answered (retrieval in the digest meta); README and skill

Goal

With task 02, the digest handles long documents itself. The API's 200-chunk refusal becomes a 5000-chunk safety cap (task 02's MaxDocChunks), and the /v1/fetch envelope's digest object says how the answer was built.

Files

• Copy: internal/api/pdf_retrieval_test.go, and the replacement internal/api/pdf_test.go (P5a's copy asserted DefaultPDFMaxQuestionChunks == 200; now 5000 — the only change; the P5b copy is the protected one from now on)
• Modify: internal/api/pdf.go (the constant), internal/api/api.go (the two digest-meta keys only), README.md, skill/SKILL.md, docs/implementer-log.md

Rules

1. DefaultPDFMaxQuestionChunks = 5000. The 422 message is unchanged.
2. In v1fetch, when the digest ran without a fallback, digMeta gains "retrieval": dr.Retrieval and "embedded": dr.Embedded, next to chunks_used and chunks_total.
3. README.md: in the PDF section, a question on a long document is answered first from BM25 (retrieval: "bm25") while its chunks are embedded in the background, and is hybrid on later questions. Also mention the 5000-chunk cap.
4. skill/SKILL.md: one sentence: --question works on long PDFs; the first answer may be bm25 (lexical); ask again later for a sharper one.

Steps

• 1. git switch p5b; copy the two given files. 2. See them fail.
• 3. The constant, the two meta keys, README, skill.
• 4. gofmt -w; go test -race -count=3 ./internal/api/ → ok. 5. make gate.
• 6. Row p5b/03-api; commit.

git add internal/api README.md skill/SKILL.md docs/implementer-log.md
git commit -F .state/commit-msg.txt

Done when

• All tests pass; make gate ok; given files byte-identical.