kyleandClaude Opus 5.5
103ec3d05a
Task 02 (the whole runner) ended at 21:08 with no file and no commit: Ornith wrote the
implementation out in its reply (~700 lines), OpenCode compacted the session, and the model
then emitted a malformed tool call. It had also looked for the given test at a wrong path
("_files/…" relative to the plan: my ambiguity).
- 02-runner → 02-runner-core (types, Submit, Status, Touch, Cancel; no goroutines) and
03-runner-workers; the given test split to match (runner_core_test.go gains three tests:
queued storage, cancel while queued, Status never moves the lease); later tasks renumbered
04-09 and every cross-reference checked.
- Copy paths spelled from the repo root.
- guard.PreCheck (+ its test) is committed here by the owner: the old task 05 told the
implementer to edit internal/guard/, which AGENTS.md forbids. My contradiction.
- AGENTS.md rule 6: write code into files early, in small steps.
Both runner test files compile against a panic-only skeleton (the core file also alone) and
fail there on panic: todo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
102 lines
3.9 KiB
Markdown
102 lines
3.9 KiB
Markdown
# P6 task 08: the `[batch]` config and `serve` wiring
|
|
|
|
**Branch:** `p6` (`git switch p6`; `git status --short` must be empty, otherwise stop)
|
|
**Commit subject:** `serve: [batch] config; open the store, reap jobs killed by a restart, start the runner`
|
|
|
|
## Goal
|
|
|
|
`sift serve` runs the batch runner from `[batch]` config. On startup it marks jobs that a restart
|
|
killed as failed (they are never resumed). On shutdown it stops the runner before closing the store.
|
|
|
|
## Files
|
|
|
|
- Copy: `docs/plans/p6/_files/config/batch_test.go` → `internal/config/batch_test.go`
|
|
- Modify: `internal/config/config.go` (the `Batch` type and field, `ParseSize`),
|
|
`cmd/sift/serve.go` (rule 3), `dist/sift.toml.example` (rule 4), `docs/implementer-log.md`
|
|
|
|
## What already exists
|
|
|
|
```go
|
|
// internal/batch (task 02)
|
|
type Config struct{ TTL time.Duration; TargetsPerJob, JobsPerCaller, JobsTotal, Workers int; MaxBytes int64; RetryWait, RetryFor time.Duration; Hooks Hooks }
|
|
func (r *Runner) Close()
|
|
|
|
// internal/api (task 06)
|
|
func (s *Server) NewBatchRunner(db *dcache.DB, cfg batch.Config) *batch.Runner
|
|
Batch *batch.Runner // in Server
|
|
// s.Prom.ObserveBatchJob(outcome string) (task 04)
|
|
|
|
// internal/dcache (task 01)
|
|
func (d *DB) MarkInFlightFailed(reason string, now int64) int
|
|
|
|
// internal/config
|
|
func ParseDuration(s string, def time.Duration) time.Duration
|
|
```
|
|
|
|
## Interfaces added here
|
|
|
|
```go
|
|
package config
|
|
|
|
type Batch struct {
|
|
Disabled bool `toml:"disabled"`
|
|
TargetsPerJob int `toml:"targets_per_job"`
|
|
JobsPerCaller int `toml:"jobs_per_caller"`
|
|
JobsTotal int `toml:"jobs_total"`
|
|
Workers int `toml:"workers"`
|
|
TTL string `toml:"ttl"`
|
|
MaxBytes string `toml:"max_bytes"`
|
|
}
|
|
|
|
// in File, after Search:
|
|
Batch Batch `toml:"batch"`
|
|
|
|
// ParseSize parses a positive byte count: a plain integer, or one with KiB, MiB or GiB.
|
|
// Anything else, zero or negative returns def.
|
|
func ParseSize(s string, def int64) int64
|
|
```
|
|
|
|
## Rules
|
|
|
|
1. Batch is **on by default**: no `[batch]` block means the runner with its own defaults.
|
|
`disabled = true` turns it off, and the endpoints then answer 503.
|
|
2. `ParseSize` trims spaces and accepts only the binary suffixes `KiB`, `MiB`, `GiB`, or none (bytes).
|
|
3. **`serve.go`:**
|
|
- Open the `dcache` store when batch is on, as well as in today's cases: same path, same call.
|
|
If the store cannot be opened, log `Warn` and run without batch.
|
|
- Then, **before** building the runner: `n := cache.MarkInFlightFailed("restart", now)`, and call
|
|
`srv.Prom.ObserveBatchJob("restart")` `n` times. Log `batch reaped` with `n` if `n > 0`.
|
|
- `srv.Batch = srv.NewBatchRunner(cache, batch.Config{...})`, with the config fields: `TTL` via
|
|
`ParseDuration(…, 0)` and `MaxBytes` via `ParseSize(…, 0)`. The runner's defaults fill the zeros.
|
|
- On shutdown (both branches of the final `select`), `srv.Batch.Close()` before the process
|
|
exits, when it is non-nil.
|
|
4. **`dist/sift.toml.example`:** a commented `[batch]` block with the defaults, in the style of the
|
|
file:
|
|
```toml
|
|
# [batch]
|
|
# disabled = false
|
|
# targets_per_job = 32
|
|
# jobs_per_caller = 1
|
|
# jobs_total = 8
|
|
# workers = 1 # at most 2: each worker fetches one target at a time
|
|
# ttl = "24h" # a job's results live this long after its last read or write
|
|
# max_bytes = "64MiB" # stored results; the oldest finished jobs go first
|
|
```
|
|
|
|
## Steps
|
|
|
|
- [ ] **1.** `git switch p6`; copy the given test. **2. See it fail.**
|
|
- [ ] **3.** `config.go`. **4.** `serve.go`. **5.** The example config.
|
|
- [ ] **6.** `gofmt -w`; `go test -race -count=3 ./internal/config/ ./cmd/sift/` → `ok`. **7.** `make gate`.
|
|
- [ ] **8.** Row `p6/08-serve-config`; commit.
|
|
|
|
```sh
|
|
git add internal/config cmd/sift/serve.go dist/sift.toml.example docs/implementer-log.md
|
|
git commit -F .state/commit-msg.txt
|
|
```
|
|
|
|
## Done when
|
|
|
|
- The given test and every existing config and cmd test pass under `-race -count=3`; `make gate`
|
|
ok; the given file is byte-identical.
|