api: questions on long PDFs are answered (retrieval in the digest meta); README and skill

The digest now handles long documents itself (task 02), so the API's refusal
limit is only a safety cap: DefaultPDFMaxQuestionChunks 200 -> 5000 (the 422
message text is unchanged). In v1fetch the digest-meta object gains "retrieval"
and "embedded" next to chunks_used/chunks_total, only when the digest ran
without a fallback. The PDF section explains a long-doc question is answered
first from BM25 while its chunks embed in the background and hybrid on later
questions; skill/SKILL.md gains one sentence. Given tests byte-identical.

Co-Authored-By: ornith-1.5-35b-a3b <ornith-1.5-35b-a3b@llama.cpp.invalid>
This commit is contained in:
2026-09-26 18:52:28 -07:00
co-authored by ornith-1.5-35b-a3b
parent f6644c9278
commit 8cdb3e8537
7 changed files with 92 additions and 8 deletions
+4 -2
View File
@@ -87,8 +87,10 @@ Answers (through `/v1/fetch` or `/v2/scrape`):
- `--question "…"` for a **digest** that, on a PDF, cites each fact as `(p. N)`.
A range returns at most 100 pages; grep shows at most 100 matches and takes a pattern of at most
200 bytes. `pages` and `grep` on anything but a PDF are a `400`. A question over a document too long
to digest (more than 200 chunks) is refused with
200 bytes. `pages` and `grep` on anything but a PDF are a `400`. A question on a long document is
answered first from BM25 (`retrieval: "bm25"`) — lexical, before the document's chunks are
embedded in the background — and hybrid on later questions, once those embeddings exist. A question
over a document too long to answer even by retrieval (more than 5000 chunks) is refused with
`422 document too long for question (N chunks); use grep or pages`.
Scanned PDFs have no text layer and return `no text layer (scanned PDF?)`; point them at an OCR