P5a review: outlines from named destinations; pool race; fast over-limit; revalidation under the limiter; Accept pdf

- pdftext/outline: resolve named destinations through the catalog /Dests and the /Names /Dests
  tree, resolve indirect /Dest, /A and /Title, and treat reader.Null (what Dict.Get returns for
  a missing key) as absent. Before this every LaTeX and Acrobat PDF had an empty outline.
  pdftest gains NamedDests; TestOutlineNamedDestinations.
- api: pool() always goes through its Once (data race on concurrent first use);
  TestDefaultPoolConcurrentFirstUse. Revalidation of a stale cache entry counts against the
  caller/host limit.
- pdftext/runner: output over MaxText kills the child at once instead of waiting for the
  deadline; TestRunnerTooMuchTextFailsFast.
- fetch: Accept includes application/pdf;q=0.8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/implementer-log.md

@@ -12,3 +12,29 @@ One row per task session that ends in a commit. Newest last.
 | p5a/06-api-pdf | 2026-09-26 | done | resumed | gate: ok (7 of 10 PDF tests; 3 are task 07's) | none | Served PDFs through `/v1/fetch` (clean) and `/v2/scrape`. Added `RetryAfter int` to `fetch.Error` and a `Retry-After` header line guarded by `> 0`. Added `PDFRunner`/`PDFPool`/`PDFTTL`/`PDFMaxQuestionChunks` Server fields (PDFPool built once via `sync.Once` only when nil, so the test's direct override is respected) and `pdfPoolOnce`. Routed `clean.Kind` in `doFetch`/`cleanFetched` (html/json/text switched, "other"→415, "pdf"→extractPDF); `doFetch`'s switch routes "text" explicitly so a PDF never runs `clean.Text` first. New `internal/api/pdf.go`: `DefaultPDFTTL`/`DefaultPDFMaxQuestionChunks` (task 07), `runner()`/`pool()` defaults, `extractPDF` (pool key `callerClass|SHA256`, two-value assert, `fmt.Errorf("pdf: bad result %T")`), `pdfError` (422 for the five pdftext errors + panic default, 503+RetryAfter 10 for ErrBusy, 429 for ErrCallerBusy), `renderPDF` (`pdfdoc.Render`, whole text or map). `doFetch` returns the 4th `doc *pdftext.Doc`; digest uses `pdfdoc.Full(doc)` (computed once) as content, clean uses `renderPDF.Content`, envelope `pdf` object has pages/title/outline/extracted/first/last/map/grep_hits/note, raw mode→400. Scrape markdown=`renderPDF.Content`, html="", `metadata.numPages`=doc.Pages. 7 of 10 given tests pass under -race -count=3; the 3 failures (`TestPDFPagesAndGrep`, `TestPDFCache`, `TestPDFQuestionTooLong`) are task 07 tests that reference fields not in this task, exactly as the task doc (lines 145, 157-158) predicts; nothing else fails. Given files byte-identical. |
 | p5a/08-digest-cli-docs | 2026-09-26 | done | 1 | gate: ok | none | Added the page-cite sentence to the digest `systemPrompt` ("When the document has `--- page N ---` markers, cite the page of each fact as (p. N).") and set `PromptVersion = "3"`. Added the `sift_pdf_extract_total{outcome}`, `sift_pdf_extract_seconds`, `sift_pdf_pages_total`, `sift_pdf_pool_wait_seconds`, `sift_pdf_cache_hits_total` series to `internal/metrics` (`pdf.go` package) and wired `ObservePDF`/`ObservePDFWait`/`PDFCacheHit` into `internal/api/pdf.go` (`ObservePDF` once per call after `extractPDF`, outcome classified by `pdfOutcome` so raw text never becomes a label; `ObservePDFWait` only when the caller actually waited on `pool().Do`; `PDFCacheHit` on fresh hits and 304s). Refactored `cmd/sift/fetchcmd.go` into a `fetchOpts` struct with `parseFetchOpts` (returns `fetchOpts`, not `string, error`) and a `fetchBody(o fetchOpts) map[string]any` that does the digest/clean default and adds `pages`/`grep` only when set; `--pages`/`--grep` added to `fetchValueFlags`. Updated `skill/SKILL.md` (PDFs section; removed PDF mention from the 415 row). Unit: `nix/module.nix` `MemoryMax = "1G"` (one line). README: added a "PDF documents" section covering detection, limits (100 MB/120 s download, 60 s extraction, 2,000 pages), the child process, the 2-slot/4-wait/20 s pool, the 24 h conditional-GET cache, the answers, the task-06 errors table, and the metrics; noted the `go-pdfkit/extract` v0.3.0 engine chosen by the `docs/P5a.md` §3 bake-off. All tests pass under -race, make gate ok, three given files byte-identical, nix diff is one line. |
 | p5a/07-api-cache-pages | 2026-09-26 | done | 1 | gate: ok | none | Implemented the PDF document cache with revalidation in `doFetch`: `pdfFromCache` (miss / fresh within `PDFTTL` / stale → `FetchIf` with 304→`TouchPDF`+answer-fresh / 200→return response with ok=false and no second GET) and `storePDF` after `extractPDF` (`ContentHash=res.SHA256`, ETag/Last-Modified from headers, FetchedAt=now, Meta=JSON of `doc.Meta`, Pages=`doc.Text`). `pages`/`grep` passed into `pdfdoc.Options` in `renderPDF`; `ErrBadPages`/`ErrBadGrep`→400 with the error's own text, non-PDF→400 "pages and grep apply to PDFs only" before the doc block. Question limit in digest mode: `n=len(digest.Split(pdfdoc.Full(doc), ChunkTokens))`, `n>PDFMaxQuestionChunks`→422 "document too long for question (n chunks); use grep or pages" before any digest call. Envelope `cached` set true for cache answers via the synthetic cache res's empty body (`len(res.Body)==0`); the one `doFetch` change beyond the rule-1 call was reusing that 200 response instead of a second `Fetch`. Added `fmt` import to api.go. Given file byte-identical; make gate ok. |
+
+## P5a review (owner, 2026-09-26)
+
+Checked: 8 task commits (`13365a2` … `7e78de6`), each with the `Co-Authored-By: ornith-1.5-35b-a3b`
+trailer; every given file byte-identical to its plan copy; `internal/guard`, `internal/auth`,
+`go.mod`, `go.sum`, `vendor/` untouched; `nix/module.nix` changed only in `MemoryMax`; `make gate`
+ok; `go test -race -count=3 ./...` ok.
+
+End-to-end with the real binary, the real guard, a real cache and dixie's 9B for digests, on
+public PDFs: Attention (15 pp, whole text, 1.9 s), GPT-3 (75 pp, map, 2.6 s), NIST SP 800-63B,
+the RP2040 datasheet (642 pp, 4.5 s), Karpathy's thesis (14.7 MB, 16 s including download).
+`pages` and `grep` from the cache in about 1.5 ms; a `question` digest with correct page
+citations (p. 1, 7, 8) in 11 s; `/v2/scrape` shape; `sift_pdf_*` metrics; no title or text in
+the logs.
+
+| # | Finding | Severity | Fault | Fix |
+|---|---|---|---|---|
+| 1 | **No outline for any LaTeX (hyperref) or Acrobat PDF** — every arXiv paper and the NIST PDF showed 0 items (qpdf: 22–120). Named destinations were skipped (task 02 said to skip them), `/Dest`/`/A`/titles were not resolved when indirect, and reader's `Dict.Get` returns `reader.Null{}` for a missing key, so the `/A` branch never ran. | high (feature absent on most papers) | task + model | owner: name-tree and `/Dests` lookup, `resolve` treats Null as absent, indirect titles; `pdftest` `NamedDests`; `TestOutlineNamedDestinations` (fails on the old code) |
+| 2 | `pool()` read `PDFPool` outside its `sync.Once` — a data race on concurrent first use with no pool configured (the production default). | medium | model | owner: always through the Once; `TestDefaultPoolConcurrentFirstUse` (race on the old code) |
+| 3 | Output over `MaxText` returned only at the deadline (the runner waited for stderr EOF while the child blocked on a full stdout pipe). | low | model | owner: check size first; `TestRunnerTooMuchTextFailsFast` (30 s on the old code) |
+| 4 | Stale-cache revalidation ran outside `acquire`, so it was not counted against the per-caller/host limit. | low | task (said "before acquire") | owner |
+| 5 | Fetch `Accept` lacked `application/pdf` (P5a §4). | low | task (not in task 01's rules) | owner |
+
+Process: 1 owner test deadlock (task 01, 55 min), 1 oversized task (06, looped 3 h, split),
+2 hangs from OpenCode 1.15 waiting on a piped stdin (driver fixed), 1 refusal-ending (task 01),
+1 malformed tool call (task 04). Every task that ran to the gate passed it first time.