api: questions on long PDFs are answered (retrieval in the digest meta); README and skill
The digest now handles long documents itself (task 02), so the API's refusal limit is only a safety cap: DefaultPDFMaxQuestionChunks 200 -> 5000 (the 422 message text is unchanged). In v1fetch the digest-meta object gains "retrieval" and "embedded" next to chunks_used/chunks_total, only when the digest ran without a fallback. The PDF section explains a long-doc question is answered first from BM25 while its chunks embed in the background and hybrid on later questions; skill/SKILL.md gains one sentence. Given tests byte-identical. Co-Authored-By: ornith-1.5-35b-a3b <ornith-1.5-35b-a3b@llama.cpp.invalid>
This commit is contained in:
@@ -87,8 +87,10 @@ Answers (through `/v1/fetch` or `/v2/scrape`):
|
||||
- `--question "…"` for a **digest** that, on a PDF, cites each fact as `(p. N)`.
|
||||
|
||||
A range returns at most 100 pages; grep shows at most 100 matches and takes a pattern of at most
|
||||
200 bytes. `pages` and `grep` on anything but a PDF are a `400`. A question over a document too long
|
||||
to digest (more than 200 chunks) is refused with
|
||||
200 bytes. `pages` and `grep` on anything but a PDF are a `400`. A question on a long document is
|
||||
answered first from BM25 (`retrieval: "bm25"`) — lexical, before the document's chunks are
|
||||
embedded in the background — and hybrid on later questions, once those embeddings exist. A question
|
||||
over a document too long to answer even by retrieval (more than 5000 chunks) is refused with
|
||||
`422 document too long for question (N chunks); use grep or pages`.
|
||||
|
||||
Scanned PDFs have no text layer and return `no text layer (scanned PDF?)`; point them at an OCR
|
||||
|
||||
Reference in New Issue
Block a user