P5a plan: PDF documents — seven Ornith tasks with acceptance tests first

docs/plans/p5a: 01 kind+fetch limits, 02 pdftext child + runner, 03 dcache PDF tables,
04 pdfpool, 05 pdfdoc render (map/pages/grep), 06 api, 07 digest citations, metrics, CLI,
skill, unit, README. Given tests compile against a panic-only skeleton and fail there for the
intended reasons; PDFs for tests are built by internal/pdftest (given), plus two qpdf-encrypted
fixtures.

Owner changes for the run: vendor go-pdfkit/extract v0.3.0 + reader v0.6.0 (pinned via
tools.go until task 02); AGENTS.md, tools/run-plan.sh (enforces a Co-Authored-By trailer for
the model), docs/implementer-log.md, `make gate`; gofmt (whitespace) on clean.go and guard;
TestUnsupportedType uses a zip now that PDFs are supported; requestDeadline 90→210 s and
writeTimeout 120→240 s for the PDF path's worst case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-25 20:47:38 -07:00
co-authored by Claude Opus 5.5
parent 3f2b78d4aa
commit cfde35397a
115 changed files with 26422 additions and 43 deletions
+81
View File
@@ -0,0 +1,81 @@
# P5a task 04: the extraction pool
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `pdfpool: a few extraction slots, a short queue, single-flight, one job per caller`
## Goal
A PDF extraction uses a child process and up to a few hundred MB, and sift shares 1 GB with it.
Kyle, 2026-09-25: "I don't think we're seeing a ton of PDF requests simultaneously, so having one
or two workers seems tenable." This task is the small in-process scheduler that enforces that:
two slots, four waiting places, identical requests sharing one job, and no single caller
holding more than one job at a time. Generic: it knows nothing about PDFs.
## Files
- Copy: `internal/pdfpool/pdfpool_test.go`
- Create: `internal/pdfpool/pdfpool.go`
- Modify: `docs/implementer-log.md`
## Interfaces
```go
package pdfpool
var (
ErrBusy = errors.New("pdf workers busy") // queue full, or waited longer than Wait
ErrCallerBusy = errors.New("busy") // this caller already has a job in flight
)
// New returns a pool with `slots` jobs running at once, at most `queue` waiting, and a waiter
// refused after `wait`.
func New(slots, queue int, wait time.Duration) *Pool
// Do runs fn under the pool and returns its result.
func (p *Pool) Do(ctx context.Context, key, caller string, fn func(context.Context) (any, error)) (any, error)
func (p *Pool) Running() int // jobs holding a slot
func (p *Pool) Queued() int // jobs waiting for a slot
func (p *Pool) Waiting(key string) int // callers sharing key's job, not counting the one that started it
```
## Rules, in the order Do applies them
1. **Join.** If a job for `key` is already running or queued, wait for its result (count it in
`Waiting(key)`) and return it. Joining takes no slot, no queue place, and is allowed even if
the caller has another job in flight. `fn` may be nil when joining.
2. **Caller.** If `caller` already has a job in flight (running or queued) for another key,
return `ErrCallerBusy` at once.
3. **Slot.** If fewer than `slots` jobs are running, start at once. Otherwise, if `queue` jobs
are already waiting, return `ErrBusy` at once. Otherwise wait, first come first served, until
a slot frees (start), `wait` passes (`ErrBusy`), or `ctx` ends (`ctx.Err()`). A waiter that
leaves gives up its place.
4. **The job's context is not the caller's.** `fn` runs with a context that is **not** cancelled
when the caller that started it goes away (`context.WithoutCancel(ctx)`): other callers may
be sharing it. The work has its own deadline (the child's). A caller whose `ctx` ends while
waiting for a result gets `ctx.Err()`; the job goes on for the others.
5. **Results are not cached.** When the job ends, the key is forgotten: the next `Do` for it runs
`fn` again. An error from `fn` reaches every sharer. A **panic** in `fn` is recovered,
returned as an error to every sharer, and frees the slot and the caller.
6. Everything is guarded by one mutex; no goroutine leaks when a waiter gives up.
## Steps
- [ ] **1.** `git switch p5a`; copy the given test. **2. See it fail** (undefined names).
- [ ] **3.** `pdfpool.go`.
- [ ] **4.** `gofmt -w`; `go test -race -count=5 ./internal/pdfpool/` → `ok`.
- [ ] **5.** `make gate`. **6.** Row `p5a/04-pdfpool`; commit.
```sh
git add internal/pdfpool docs/implementer-log.md
git commit -F .state/commit-msg.txt
```
## Done when
- The given test passes under `-race -count=5`; `make gate` ok; given file byte-identical.
## Stop and report if
- A given test's timing looks impossible to meet without sleeping in production code (that is
the owner's bug, per `AGENTS.md`).