36 lines
2.2 KiB
Markdown
36 lines
2.2 KiB
Markdown
# P5b implementation plan: questions on long documents
|
||
|
||
> **For the implementing model:** do not work from this file. The owner gives you one task file
|
||
> at a time. This file is the index for the owner and the reviewer.
|
||
|
||
**Goal:** P5a refuses `question` on a PDF longer than 200 digest chunks (~240k tokens) with
|
||
"use grep or pages". After P5b such a question is answered: at once from **BM25** over
|
||
page-aware chunks, while the document's chunks are **embedded in the background**; once they
|
||
are, later questions use today's hybrid score. Two facts found while planning:
|
||
|
||
- The embedding cache (`dcache` `GetEmbed`/`PutEmbed`, `EmbedKey`) exists but **nothing uses it**:
|
||
every question re-embeds every chunk.
|
||
- `Split` is line-based, so a chunk that starts mid-page carries no `--- page N ---` marker and
|
||
the model cannot cite its page.
|
||
|
||
| Task | Package | Given |
|
||
|---|---|---|
|
||
| 01-splitpages-bm25 | `internal/digest` (pure functions) | `digest/splitpages_test.go`, `digest/bm25_test.go` |
|
||
| 02-retrieval | `internal/digest` | `digest/retrieval_test.go` |
|
||
| 03-api | `internal/api`, `README.md`, `skill/SKILL.md` | `api/pdf_retrieval_test.go`; replaces `api/pdf_test.go` (P5a: the 200-chunk default becomes 5000) |
|
||
|
||
**How this plan was made:** acceptance tests first, no reference implementation; the given tests
|
||
compiled against a panic-only skeleton on master `1311cf3` and failed for the intended reasons.
|
||
|
||
## Changes during the run
|
||
|
||
- **Run 1, task 01 (2026-09-26 11:01–12:24): infrastructure, not the model.** straylight's
|
||
llama-server ran out of shared KV (4 slots over one 262,144-token pool). The session got
|
||
`Context size has been exceeded` with no commit. Hermes's ornith context was capped at 131072
|
||
(Kyle: "b is acceptable"); the task was re-run clean and passed (`dca0004`, rebased to `7595ad8`).
|
||
- **Task 02, first attempt: my test, not the model or the task.** `TestEmbeddingsAreCachedAcrossQuestions`
|
||
used `longDoc(12)`, which is 4,617 tokens. That is under `SmallDoc` (6000), so rule 2 answers
|
||
it `"whole"`, while the test expected `"hybrid"`. Ornith measured the document and was working
|
||
it out when I stopped the session (22 min). Fix `d52a4fa`: `longDoc(30)` (11,677 tokens, 10
|
||
chunks) plus a `t.Fatalf` precondition. Relaunched from task 02 on a clean tree.
|