kyleandClaude Opus 5.5
103ec3d05a
Task 02 (the whole runner) ended at 21:08 with no file and no commit: Ornith wrote the
implementation out in its reply (~700 lines), OpenCode compacted the session, and the model
then emitted a malformed tool call. It had also looked for the given test at a wrong path
("_files/…" relative to the plan: my ambiguity).
- 02-runner → 02-runner-core (types, Submit, Status, Touch, Cancel; no goroutines) and
03-runner-workers; the given test split to match (runner_core_test.go gains three tests:
queued storage, cancel while queued, Status never moves the lease); later tasks renumbered
04-09 and every cross-reference checked.
- Copy paths spelled from the repo root.
- guard.PreCheck (+ its test) is committed here by the owner: the old task 05 told the
implementer to edit internal/guard/, which AGENTS.md forbids. My contradiction.
- AGENTS.md rule 6: write code into files early, in small steps.
Both runner test files compile against a panic-only skeleton (the core file also alone) and
fail there on panic: todo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
87 lines
4.5 KiB
Markdown
87 lines
4.5 KiB
Markdown
# P6 task 03: the batch runner's workers
|
||
|
||
**Branch:** `p6` (`git switch p6`; `git status --short` must be empty, otherwise stop)
|
||
**Commit subject:** `batch: workers — round-robin, serial per job, busy retry, cancel in flight, hooks`
|
||
|
||
## Goal
|
||
|
||
Task 02 wrote the runner's types, `Submit`, `Status`, `Touch` and `Cancel`; every job stays
|
||
`queued`. This task adds the workers that drain jobs: one target at a time per job, round-robin
|
||
across jobs, with a retry for a target that was only busy.
|
||
|
||
## Files
|
||
|
||
- Copy: `docs/plans/p6/_files/batch/runner_test.go` → `internal/batch/runner_test.go`
|
||
- Create: `internal/batch/worker.go` (the workers, the rotation, one target, the terminal status)
|
||
- Modify: `internal/batch/batch.go` (`New` starts the workers; `Close` stops them; `Submit` wakes
|
||
one; `Cancel` handles a job with a target in flight), `docs/implementer-log.md`
|
||
|
||
**Work in small steps, in files.** Write `worker.go` with the worker loop and a stub for one
|
||
target, and make it compile. Then fill it in, running the tests as you go. Do not write the
|
||
implementation out in your reply before putting it in a file.
|
||
|
||
## What already exists (task 02, in `internal/batch/batch.go`)
|
||
|
||
All the types in the task 02 file: `Options`, `Target`, `Result`, `TargetError`, `FetchFunc`,
|
||
`Job`, `TargetStatus`, `Hooks`, `Config`, the errors (including `ErrRetryLater`), `Runner`,
|
||
`New`, `Submit`, `Status`, `Touch`, `Cancel`, `Close`. Read `batch.go` for how the runner keeps
|
||
its jobs and each job's caller; the dcache API it uses is listed in the task 02 file's
|
||
"What already exists".
|
||
|
||
## Rules
|
||
|
||
1. **Workers.** `Workers` goroutines share one rotation of live jobs in submit order. A worker
|
||
takes the next job in the rotation that has a pending target **and no target in flight**. It
|
||
sets the job to `running` on its first target, fetches that one target, stores it, and moves
|
||
on to the next job in the rotation. So:
|
||
- each job is serial: its target `i+1` never starts before `i` is stored;
|
||
- with one worker and two live jobs, their targets alternate (`a1 b1 a2 b2 …`);
|
||
- a job that a worker has not reached yet stays `queued`.
|
||
2. **One target:**
|
||
- Call the `FetchFunc` with the job's caller, under a context that `Close` cancels. Recover
|
||
a panic into the error `internal error` (never echo the panic value).
|
||
- If the error wraps `ErrRetryLater`, the target stays `pending`: the worker goes on with the
|
||
rotation, and the target is tried again after `RetryWait`. A target that has been coming
|
||
back busy for longer than `RetryFor` is stored `failed` with the error's text.
|
||
- Otherwise store it: success → `done` with the JSON of the `Result`; error → `failed`, with
|
||
`TargetError`'s `Msg`/`Hint` or `err.Error()` and no hint.
|
||
- Then call `Hooks.Target(err, time, len(result JSON))`, then `EvictBatchOldest(MaxBytes)`.
|
||
3. **Terminal status.** When a job's last target is stored: all `done` → `done`; all `failed` →
|
||
`failed`; a mix → `partial`. `SetBatchJobStatus`, then `Hooks.Job(status)` and
|
||
`Hooks.JobSeconds(now − created)`, exactly once per job.
|
||
4. **Cancel(id)** returns false for an unknown id or a job already terminal. Otherwise it sets the job
|
||
`cancelled` at once. The in-flight target, if any, finishes and is stored; no further target
|
||
starts; later targets stay `pending`. Storing that last target must not overwrite `cancelled`.
|
||
`Hooks.Job("cancelled")` once.
|
||
5. **Hooks.Depth** gets the live-job count and **Hooks.Inflight** the in-flight count whenever
|
||
either changes.
|
||
|
||
## Facts
|
||
|
||
- The given test uses `FetchFunc`s that hold on channels, fail, return `ErrRetryLater`, or panic.
|
||
It calls `Close` before the store closes.
|
||
- `TestRoundRobin` holds the first target until the second job is submitted, then expects
|
||
`a1 b1 a2 b2 a3 b3` from one worker.
|
||
- Task 02's tests (`runner_core_test.go`) must still pass: with workers, a job there may
|
||
already be `running` or `done` where it was `queued`; those tests allow for that.
|
||
|
||
## Steps
|
||
|
||
- [ ] **1.** `git switch p6`; copy the given test. **2. See it fail** (the jobs never finish).
|
||
- [ ] **3.** `worker.go`. **4.** The `batch.go` changes.
|
||
- [ ] **5.** `gofmt -w`; `go test -race -count=3 ./internal/batch/` → `ok`. **6.** `make gate`.
|
||
- [ ] **7.** Row `p6/03-runner-workers`; commit.
|
||
|
||
```sh
|
||
git add internal/batch docs/implementer-log.md
|
||
git commit -F .state/commit-msg.txt
|
||
```
|
||
|
||
## Done when
|
||
|
||
- Both given tests pass under `-race -count=3`; `make gate` ok; the given files are byte-identical.
|
||
|
||
## Stop and report if
|
||
|
||
- A given test's timing can only be met by sleeping longer than `RetryWait` inside a worker.
|