# P6 task 08: the `[batch]` config and `serve` wiring **Branch:** `p6` (`git switch p6`; `git status --short` must be empty, otherwise stop) **Commit subject:** `serve: [batch] config; open the store, reap jobs killed by a restart, start the runner` ## Goal `sift serve` runs the batch runner from `[batch]` config. On startup it marks jobs that a restart killed as failed (they are never resumed). On shutdown it stops the runner before closing the store. ## Files - Copy: `docs/plans/p6/_files/config/batch_test.go` → `internal/config/batch_test.go` - Modify: `internal/config/config.go` (the `Batch` type and field, `ParseSize`), `cmd/sift/serve.go` (rule 3), `dist/sift.toml.example` (rule 4), `docs/implementer-log.md` ## What already exists ```go // internal/batch (task 02) type Config struct{ TTL time.Duration; TargetsPerJob, JobsPerCaller, JobsTotal, Workers int; MaxBytes int64; RetryWait, RetryFor time.Duration; Hooks Hooks } func (r *Runner) Close() // internal/api (task 06) func (s *Server) NewBatchRunner(db *dcache.DB, cfg batch.Config) *batch.Runner Batch *batch.Runner // in Server // s.Prom.ObserveBatchJob(outcome string) (task 04) // internal/dcache (task 01) func (d *DB) MarkInFlightFailed(reason string, now int64) int // internal/config func ParseDuration(s string, def time.Duration) time.Duration ``` ## Interfaces added here ```go package config type Batch struct { Disabled bool `toml:"disabled"` TargetsPerJob int `toml:"targets_per_job"` JobsPerCaller int `toml:"jobs_per_caller"` JobsTotal int `toml:"jobs_total"` Workers int `toml:"workers"` TTL string `toml:"ttl"` MaxBytes string `toml:"max_bytes"` } // in File, after Search: Batch Batch `toml:"batch"` // ParseSize parses a positive byte count: a plain integer, or one with KiB, MiB or GiB. // Anything else, zero or negative returns def. func ParseSize(s string, def int64) int64 ``` ## Rules 1. Batch is **on by default**: no `[batch]` block means the runner with its own defaults. `disabled = true` turns it off, and the endpoints then answer 503. 2. `ParseSize` trims spaces and accepts only the binary suffixes `KiB`, `MiB`, `GiB`, or none (bytes). 3. **`serve.go`:** - Open the `dcache` store when batch is on, as well as in today's cases: same path, same call. If the store cannot be opened, log `Warn` and run without batch. - Then, **before** building the runner: `n := cache.MarkInFlightFailed("restart", now)`, and call `srv.Prom.ObserveBatchJob("restart")` `n` times. Log `batch reaped` with `n` if `n > 0`. - `srv.Batch = srv.NewBatchRunner(cache, batch.Config{...})`, with the config fields: `TTL` via `ParseDuration(…, 0)` and `MaxBytes` via `ParseSize(…, 0)`. The runner's defaults fill the zeros. - On shutdown (both branches of the final `select`), `srv.Batch.Close()` before the process exits, when it is non-nil. 4. **`dist/sift.toml.example`:** a commented `[batch]` block with the defaults, in the style of the file: ```toml # [batch] # disabled = false # targets_per_job = 32 # jobs_per_caller = 1 # jobs_total = 8 # workers = 1 # at most 2: each worker fetches one target at a time # ttl = "24h" # a job's results live this long after its last read or write # max_bytes = "64MiB" # stored results; the oldest finished jobs go first ``` ## Steps - [ ] **1.** `git switch p6`; copy the given test. **2. See it fail.** - [ ] **3.** `config.go`. **4.** `serve.go`. **5.** The example config. - [ ] **6.** `gofmt -w`; `go test -race -count=3 ./internal/config/ ./cmd/sift/` → `ok`. **7.** `make gate`. - [ ] **8.** Row `p6/08-serve-config`; commit. ```sh git add internal/config cmd/sift/serve.go dist/sift.toml.example docs/implementer-log.md git commit -F .state/commit-msg.txt ``` ## Done when - The given test and every existing config and cmd test pass under `-race -count=3`; `make gate` ok; the given file is byte-identical.