From 583a35883d8569caefd4c4324e0f1b55076bae0b Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Sat, 26 Sep 2026 20:34:57 -0700 Subject: [PATCH] =?UTF-8?q?PLAN:=20future=20work=20=E2=80=94=20digest=20re?= =?UTF-8?q?trieval=20misses=20terse=20summary=20facts=20(RP2040=20PIO=20co?= =?UTF-8?q?unt,=20prod=20check=20after=20P5b)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- docs/PLAN.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/PLAN.md b/docs/PLAN.md index c505340..2bddaf6 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -313,6 +313,19 @@ its acceptance evidence posted in `~agent`. ## 13. Future work +### Digest retrieval misses terse summary facts (found 2026-09-26, after P5b) + +On the RP2040 datasheet, "how many PIO state machines in total?" came back **4** from both +`bm25` and `hybrid` retrieval. The answer, **8**, is a one-line bullet on p. 9 ("8 PIO state +machines") that neither ranker put in the top 5 chunks, so the model answered from the +per-block sections, and in the hybrid case also made up a split. `--grep` finds it at once, and +the skill now says to confirm numbers with `--grep` (`94da9d9`). + +Candidates, when there is a phase for it: +- pin the chunk holding the document's feature or overview list, as chunk 0 is pinned today; +- run a `--grep`-style exact-term pass over the question's numbers and nouns before ranking; +- a small eval set of fact questions over the pdfbench corpus, to measure a change rather than guess. + ### Paywalled pages → archive snapshot (Kyle, 2026-09-26; possible P7) "If a site is paywalled, we may be able to use archive.is to fetch it."