P6 during the run: split the runner task (02 core, 03 workers); guard.PreCheck as an owner commit; AGENTS.md: write code into files early
Task 02 (the whole runner) ended at 21:08 with no file and no commit: Ornith wrote the
implementation out in its reply (~700 lines), OpenCode compacted the session, and the model
then emitted a malformed tool call. It had also looked for the given test at a wrong path
("_files/…" relative to the plan: my ambiguity).
- 02-runner → 02-runner-core (types, Submit, Status, Touch, Cancel; no goroutines) and
03-runner-workers; the given test split to match (runner_core_test.go gains three tests:
queued storage, cancel while queued, Status never moves the lease); later tasks renumbered
04-09 and every cross-reference checked.
- Copy paths spelled from the repo root.
- guard.PreCheck (+ its test) is committed here by the owner: the old task 05 told the
implementer to edit internal/guard/, which AGENTS.md forbids. My contradiction.
- AGENTS.md rule 6: write code into files early, in small steps.
Both runner test files compile against a panic-only skeleton (the core file also alone) and
fail there on panic: todo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
# P6 task 08: the `[batch]` config and `serve` wiring
|
||||
|
||||
**Branch:** `p6` (`git switch p6`; `git status --short` must be empty, otherwise stop)
|
||||
**Commit subject:** `serve: [batch] config; open the store, reap jobs killed by a restart, start the runner`
|
||||
|
||||
## Goal
|
||||
|
||||
`sift serve` runs the batch runner from `[batch]` config. On startup it marks jobs that a restart
|
||||
killed as failed (they are never resumed). On shutdown it stops the runner before closing the store.
|
||||
|
||||
## Files
|
||||
|
||||
- Copy: `docs/plans/p6/_files/config/batch_test.go` → `internal/config/batch_test.go`
|
||||
- Modify: `internal/config/config.go` (the `Batch` type and field, `ParseSize`),
|
||||
`cmd/sift/serve.go` (rule 3), `dist/sift.toml.example` (rule 4), `docs/implementer-log.md`
|
||||
|
||||
## What already exists
|
||||
|
||||
```go
|
||||
// internal/batch (task 02)
|
||||
type Config struct{ TTL time.Duration; TargetsPerJob, JobsPerCaller, JobsTotal, Workers int; MaxBytes int64; RetryWait, RetryFor time.Duration; Hooks Hooks }
|
||||
func (r *Runner) Close()
|
||||
|
||||
// internal/api (task 06)
|
||||
func (s *Server) NewBatchRunner(db *dcache.DB, cfg batch.Config) *batch.Runner
|
||||
Batch *batch.Runner // in Server
|
||||
// s.Prom.ObserveBatchJob(outcome string) (task 04)
|
||||
|
||||
// internal/dcache (task 01)
|
||||
func (d *DB) MarkInFlightFailed(reason string, now int64) int
|
||||
|
||||
// internal/config
|
||||
func ParseDuration(s string, def time.Duration) time.Duration
|
||||
```
|
||||
|
||||
## Interfaces added here
|
||||
|
||||
```go
|
||||
package config
|
||||
|
||||
type Batch struct {
|
||||
Disabled bool `toml:"disabled"`
|
||||
TargetsPerJob int `toml:"targets_per_job"`
|
||||
JobsPerCaller int `toml:"jobs_per_caller"`
|
||||
JobsTotal int `toml:"jobs_total"`
|
||||
Workers int `toml:"workers"`
|
||||
TTL string `toml:"ttl"`
|
||||
MaxBytes string `toml:"max_bytes"`
|
||||
}
|
||||
|
||||
// in File, after Search:
|
||||
Batch Batch `toml:"batch"`
|
||||
|
||||
// ParseSize parses a positive byte count: a plain integer, or one with KiB, MiB or GiB.
|
||||
// Anything else, zero or negative returns def.
|
||||
func ParseSize(s string, def int64) int64
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
1. Batch is **on by default**: no `[batch]` block means the runner with its own defaults.
|
||||
`disabled = true` turns it off, and the endpoints then answer 503.
|
||||
2. `ParseSize` trims spaces and accepts only the binary suffixes `KiB`, `MiB`, `GiB`, or none (bytes).
|
||||
3. **`serve.go`:**
|
||||
- Open the `dcache` store when batch is on, as well as in today's cases: same path, same call.
|
||||
If the store cannot be opened, log `Warn` and run without batch.
|
||||
- Then, **before** building the runner: `n := cache.MarkInFlightFailed("restart", now)`, and call
|
||||
`srv.Prom.ObserveBatchJob("restart")` `n` times. Log `batch reaped` with `n` if `n > 0`.
|
||||
- `srv.Batch = srv.NewBatchRunner(cache, batch.Config{...})`, with the config fields: `TTL` via
|
||||
`ParseDuration(…, 0)` and `MaxBytes` via `ParseSize(…, 0)`. The runner's defaults fill the zeros.
|
||||
- On shutdown (both branches of the final `select`), `srv.Batch.Close()` before the process
|
||||
exits, when it is non-nil.
|
||||
4. **`dist/sift.toml.example`:** a commented `[batch]` block with the defaults, in the style of the
|
||||
file:
|
||||
```toml
|
||||
# [batch]
|
||||
# disabled = false
|
||||
# targets_per_job = 32
|
||||
# jobs_per_caller = 1
|
||||
# jobs_total = 8
|
||||
# workers = 1 # at most 2: each worker fetches one target at a time
|
||||
# ttl = "24h" # a job's results live this long after its last read or write
|
||||
# max_bytes = "64MiB" # stored results; the oldest finished jobs go first
|
||||
```
|
||||
|
||||
## Steps
|
||||
|
||||
- [ ] **1.** `git switch p6`; copy the given test. **2. See it fail.**
|
||||
- [ ] **3.** `config.go`. **4.** `serve.go`. **5.** The example config.
|
||||
- [ ] **6.** `gofmt -w`; `go test -race -count=3 ./internal/config/ ./cmd/sift/` → `ok`. **7.** `make gate`.
|
||||
- [ ] **8.** Row `p6/08-serve-config`; commit.
|
||||
|
||||
```sh
|
||||
git add internal/config cmd/sift/serve.go dist/sift.toml.example docs/implementer-log.md
|
||||
git commit -F .state/commit-msg.txt
|
||||
```
|
||||
|
||||
## Done when
|
||||
|
||||
- The given test and every existing config and cmd test pass under `-race -count=3`; `make gate`
|
||||
ok; the given file is byte-identical.
|
||||
Reference in New Issue
Block a user