diff --git a/docs/PLAN.md b/docs/PLAN.md index c505340..2bddaf6 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -313,6 +313,19 @@ its acceptance evidence posted in `~agent`. ## 13. Future work +### Digest retrieval misses terse summary facts (found 2026-09-26, after P5b) + +On the RP2040 datasheet, "how many PIO state machines in total?" came back **4** from both +`bm25` and `hybrid` retrieval. The answer, **8**, is a one-line bullet on p. 9 ("8 PIO state +machines") that neither ranker put in the top 5 chunks, so the model answered from the +per-block sections, and in the hybrid case also made up a split. `--grep` finds it at once, and +the skill now says to confirm numbers with `--grep` (`94da9d9`). + +Candidates, when there is a phase for it: +- pin the chunk holding the document's feature or overview list, as chunk 0 is pinned today; +- run a `--grep`-style exact-term pass over the question's numbers and nouns before ranking; +- a small eval set of fact questions over the pdfbench corpus, to measure a change rather than guess. + ### Paywalled pages → archive snapshot (Kyle, 2026-09-26; possible P7) "If a site is paywalled, we may be able to use archive.is to fetch it."