sift/docs/plans/p5a/04-pdfpool.md

P5a task 04: the extraction pool

Branch: p5a (git switch p5a; git status --short must be empty, otherwise stop) Commit subject: pdfpool: a few extraction slots, a short queue, single-flight, one job per caller

Goal

A PDF extraction uses a child process and up to a few hundred MB, and sift shares 1 GB with it. Kyle, 2026-09-25: "I don't think we're seeing a ton of PDF requests simultaneously, so having one or two workers seems tenable." This task is the small in-process scheduler that enforces that: two slots, four waiting places, identical requests sharing one job, and no single caller holding more than one job at a time. Generic: it knows nothing about PDFs.

Files

• Copy: internal/pdfpool/pdfpool_test.go
• Create: internal/pdfpool/pdfpool.go
• Modify: docs/implementer-log.md

Interfaces

package pdfpool

var (
	ErrBusy       = errors.New("pdf workers busy") // queue full, or waited longer than Wait
	ErrCallerBusy = errors.New("busy")             // this caller already has a job in flight
)

// New returns a pool with `slots` jobs running at once, at most `queue` waiting, and a waiter
// refused after `wait`.
func New(slots, queue int, wait time.Duration) *Pool

// Do runs fn under the pool and returns its result.
func (p *Pool) Do(ctx context.Context, key, caller string, fn func(context.Context) (any, error)) (any, error)

func (p *Pool) Running() int           // jobs holding a slot
func (p *Pool) Queued() int            // jobs waiting for a slot
func (p *Pool) Waiting(key string) int // callers sharing key's job, not counting the one that started it

Rules, in the order Do applies them

1. Join. If a job for key is already running or queued, wait for its result (count it in Waiting(key)) and return it. Joining takes no slot, no queue place, and is allowed even if the caller has another job in flight. fn may be nil when joining.
2. Caller. If caller already has a job in flight (running or queued) for another key, return ErrCallerBusy at once.
3. Slot. If fewer than slots jobs are running, start at once. Otherwise, if queue jobs are already waiting, return ErrBusy at once. Otherwise wait, first come first served, until a slot frees (start), wait passes (ErrBusy), or ctx ends (ctx.Err()). A waiter that leaves gives up its place.
4. The job's context is not the caller's. fn runs with a context that is not cancelled when the caller that started it goes away (context.WithoutCancel(ctx)): other callers may be sharing it. The work has its own deadline (the child's). A caller whose ctx ends while waiting for a result gets ctx.Err(); the job goes on for the others.
5. Results are not cached. When the job ends, the key is forgotten: the next Do for it runs fn again. An error from fn reaches every sharer. A panic in fn is recovered, returned as an error to every sharer, and frees the slot and the caller.
6. Everything is guarded by one mutex; no goroutine leaks when a waiter gives up.

Steps

• 1. git switch p5a; copy the given test. 2. See it fail (undefined names).
• 3. pdfpool.go.
• 4. gofmt -w; go test -race -count=5 ./internal/pdfpool/ → ok.
• 5. make gate. 6. Row p5a/04-pdfpool; commit.

git add internal/pdfpool docs/implementer-log.md
git commit -F .state/commit-msg.txt

Done when

• The given test passes under -race -count=5; make gate ok; given file byte-identical.

Stop and report if

• A given test's timing looks impossible to meet without sleeping in production code (that is the owner's bug, per AGENTS.md).