P5a plan: PDF documents — seven Ornith tasks with acceptance tests first

docs/plans/p5a: 01 kind+fetch limits, 02 pdftext child + runner, 03 dcache PDF tables,
04 pdfpool, 05 pdfdoc render (map/pages/grep), 06 api, 07 digest citations, metrics, CLI,
skill, unit, README. Given tests compile against a panic-only skeleton and fail there for the
intended reasons; PDFs for tests are built by internal/pdftest (given), plus two qpdf-encrypted
fixtures.

Owner changes for the run: vendor go-pdfkit/extract v0.3.0 + reader v0.6.0 (pinned via
tools.go until task 02); AGENTS.md, tools/run-plan.sh (enforces a Co-Authored-By trailer for
the model), docs/implementer-log.md, `make gate`; gofmt (whitespace) on clean.go and guard;
TestUnsupportedType uses a zip now that PDFs are supported; requestDeadline 90→210 s and
writeTimeout 120→240 s for the PDF path's worst case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-25 20:47:38 -07:00
co-authored by Claude Opus 5.5
parent 3f2b78d4aa
commit cfde35397a
115 changed files with 26422 additions and 43 deletions
+115
View File
@@ -0,0 +1,115 @@
# P5a task 05: render an extracted document for an agent
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `pdfdoc: page markers, the map for long documents, page ranges, grep`
## Goal
An agent's context is small and a thesis is 60k–150k tokens. This task is the pure function that
decides what an agent sees: the whole text with page markers when it fits; otherwise a **map**
(title, outline with page numbers, the first pages, and how to ask for more); a **page range**;
or **grep** results with page numbers. No I/O, no PDFs: it works on a `*pdftext.Doc`.
## Files
- Copy: `internal/pdfdoc/pdfdoc_test.go`
- Create: `internal/pdfdoc/pdfdoc.go` (`Render`, `Full`, `Marker`, the map, page ranges),
`internal/pdfdoc/grep.go`
- Modify: `docs/implementer-log.md`
## Interfaces
```go
package pdfdoc
type Options struct {
Pages string // "" = no range; "a-b", "a" or "a-" (1-based, inclusive)
Grep string // "" = no grep; an RE2 pattern
OutCap int // characters (runes) the content may use
}
type Rendered struct {
Content string
Map bool // Content is the map of a document longer than OutCap
First, Last int // pages whose text is in Content; 0, 0 for a grep answer
GrepHits int // matches found (more may exist than are shown)
Note string // "" or how the answer was cut and how to continue
}
var (
ErrBadPages = errors.New("invalid pages")
ErrBadGrep = errors.New("invalid grep")
)
func Marker(page int) string // "--- page N ---"
func Full(doc *pdftext.Doc) string // every extracted page with its marker
func Render(doc *pdftext.Doc, opt Options) (Rendered, error)
```
## Rules
1. **Full.** For each extracted page p: `Marker(p) + "\n\n" + strings.TrimSpace(text)`, joined
with `"\n\n"`. The given test shows the exact string.
2. **No Pages, no Grep.**
- If the rune count of `Full(doc)` is at most `OutCap`: `Content = Full(doc)`, `First = 1`,
`Last` = pages extracted.
- Otherwise `Map = true` and the content is, in this order:
```
# <Title, or "Untitled PDF">
<Pages> pages · ~<T> tokens[ · pages 1-<E> extracted]
## Outline (only when the outline is not empty)
- <title> (p. <page>) (one line per item, indented two spaces per level below 1)
This document is longer than one answer: ask for pages="a-b" or grep="pattern".
<leading pages>
```
- `T` is the rune count of `Full(doc)` divided by 4.
- The `pages 1-E extracted` part appears only when fewer pages were extracted than
`Pages`. The separator is " · " (U+00B7).
- Leading pages are whole pages rendered as in `Full`, from page 1, as many as fit in
`OutCap/4` runes. At least page 1 is shown, cut to fit if it alone is larger.
- The whole content is at most `OutCap` runes; if the outline is too long, cut the outline,
not the header or the instruction line.
- `First = 1`, `Last` = the last whole page shown.
3. **Pages.**
- Parse after trimming spaces: `a-b`, `a`, `a-` (a to the last extracted page). Anything else
→ `ErrBadPages`.
- `a < 1`, `b < a`, or `a` past the last extracted page → an error wrapping `ErrBadPages`
whose text is `pages out of range (1-E)`. `b` past the end is clamped to `E`.
- At most **100 pages** and at most `OutCap` runes: whole pages from `a`. A single page larger
than `OutCap` is cut to fit.
- When the range was cut, `Note` says so and names the rest as `pages="k-b"`: for example
`cut at page 13; ask for pages="14-40"`.
4. **Grep.**
- The pattern must be at most 200 bytes and compile with `regexp` (RE2), else an error
wrapping `ErrBadGrep`. Prefix `(?i)` unless the pattern contains an upper-case letter
("smart case").
- Search every line of every page in the range (all extracted pages when `Pages` is empty).
- The first line of the content is `grep "<pattern>": N matches` (`1 match` for one; `no matches`
for none). Add `, showing the first 100` when N is over 100.
- Then, per match, up to the first 100: the previous line of the same page as
`p. P- <line>`, the matching line as `p. P: <line>`, the next line as `p. P- <line>`,
then a `--` line. Lines are trimmed of surrounding spaces.
- `GrepHits = N`, `First = Last = 0`. The content is at most `OutCap` runes (drop whole
match blocks at the end and say so in `Note`). No match is not an error.
5. Every length here is in **runes**, never bytes: the cap protects an agent's context, and CJK
text is three bytes a rune.
## Steps
- [ ] **1.** `git switch p5a`; copy the given test. **2. See it fail** (undefined names).
- [ ] **3.** `pdfdoc.go`, **4.** `grep.go`.
- [ ] **5.** `gofmt -w`; `go test -race -count=3 ./internal/pdfdoc/` → `ok`.
- [ ] **6.** `make gate`. **7.** Row `p5a/05-pdfdoc`; commit.
```sh
git add internal/pdfdoc docs/implementer-log.md
git commit -F .state/commit-msg.txt
```
## Done when
- The given test passes; `make gate` ok; given file byte-identical.