44 lines
2.0 KiB
Markdown
44 lines
2.0 KiB
Markdown
# P5b task 03: the API answers long-document questions and says how
|
|
|
|
**Branch:** `p5b` (`git switch p5b`; `git status --short` must be empty, otherwise stop)
|
|
**Commit subject:** `api: questions on long PDFs are answered (retrieval in the digest meta); README and skill`
|
|
|
|
## Goal
|
|
|
|
With task 02, the digest handles long documents itself. The API's 200-chunk refusal becomes a
|
|
5000-chunk safety cap (task 02's `MaxDocChunks`), and the `/v1/fetch` envelope's `digest`
|
|
object says how the answer was built.
|
|
|
|
## Files
|
|
|
|
- Copy: `internal/api/pdf_retrieval_test.go`, and the **replacement** `internal/api/pdf_test.go`
|
|
(P5a's copy asserted `DefaultPDFMaxQuestionChunks == 200`; now 5000 — the only change; the
|
|
P5b copy is the protected one from now on)
|
|
- Modify: `internal/api/pdf.go` (the constant), `internal/api/api.go` (the two digest-meta keys
|
|
only), `README.md`, `skill/SKILL.md`, `docs/implementer-log.md`
|
|
|
|
## Rules
|
|
|
|
1. `DefaultPDFMaxQuestionChunks = 5000`. The 422 message is unchanged.
|
|
2. In `v1fetch`, when the digest ran without a fallback, `digMeta` gains
|
|
`"retrieval": dr.Retrieval` and `"embedded": dr.Embedded`, next to `chunks_used` and
|
|
`chunks_total`.
|
|
3. **`README.md`:** in the PDF section, a question on a long document is answered first from BM25 (`retrieval: "bm25"`) while its chunks are embedded in the background, and is hybrid on later questions. Also mention the 5000-chunk cap.
|
|
4. **`skill/SKILL.md`:** one sentence: `--question` works on long PDFs; the first answer may be `bm25` (lexical); ask again later for a sharper one.
|
|
|
|
## Steps
|
|
|
|
- [ ] **1.** `git switch p5b`; copy the two given files. **2. See them fail.**
|
|
- [ ] **3.** The constant, the two meta keys, README, skill.
|
|
- [ ] **4.** `gofmt -w`; `go test -race -count=3 ./internal/api/` → `ok`. **5.** `make gate`.
|
|
- [ ] **6.** Row `p5b/03-api`; commit.
|
|
|
|
```sh
|
|
git add internal/api README.md skill/SKILL.md docs/implementer-log.md
|
|
git commit -F .state/commit-msg.txt
|
|
```
|
|
|
|
## Done when
|
|
|
|
- All tests pass; `make gate` ok; given files byte-identical.
|