sift/docs/plans/p6/06-api-endpoints.md

P6 task 06: the /v1/batch endpoints

Branch: p6 (git switch p6; git status --short must be empty, otherwise stop) Commit subject: api: /v1/batch — submit, status summary, one result, cancel

Goal

The HTTP surface over s.Batch (task 05): submit, a status summary, one target's result, and cancel, with the submit-time guard pre-check, the caps as HTTP statuses, and ownership.

Files

• Copy: _files/api/batch_test.go → internal/api/batch_test.go
• Modify: internal/api/batch.go (the four handlers), internal/api/api.go (only the routes in New and ServeHTTP, rule 2), docs/implementer-log.md

What already exists

// internal/batch (task 02)
type Target struct{ URL string; Options Options }
type Options struct{ Mode, Question, Pages, Grep, JQ string; Context *bool; Recover bool }
type Result struct{ Content string; Meta map[string]any }
type Job struct{ ID, Caller, Status, Error string; TargetsTotal, TargetsDone int; CreatedAt int64 }
type TargetStatus struct{ Idx int; URL, Status string; Result *Result; Error, Hint string }
var ErrNoTargets, ErrTooManyTargets, ErrCallerBusy, ErrBusy error
func (r *Runner) Submit(ctx context.Context, c *auth.Caller, callerClass string, targets []Target) (string, bool, error)
func (r *Runner) Cancel(id string) bool
func (r *Runner) Status(id string) (Job, []TargetStatus, bool) // a pure read
func (r *Runner) Touch(id string)                              // bumps the lease to now

// internal/guard (task 05)
func (g *Guard) PreCheck(raw string) error // "blocked: …" errors

// internal/api
	Batch *batch.Runner // in Server (task 05); nil → 503 "batch disabled"
func (s *Server) caller(w http.ResponseWriter, r *http.Request) *auth.Caller // nil after writing 401/403
func (s *Server) callerClass(c *auth.Caller) string
// maxReqBody (64 KiB), writeErr(w, status, msg, hint), writeJSON(w, status, v), s.Prom.Observe(path, d, code)

Add const batchMaxBody = 256 << 10.

Rules

1. Routes (in New): POST /v1/batch, GET /v1/batch/{id}, GET /v1/batch/{id}/{idx}, DELETE /v1/batch/{id}. Each starts with s.caller(w, r).
2. ServeHTTP:
   • The body cap is batchMaxBody for /v1/batch and maxReqBody for every other path. The cap is applied once, here: a handler cannot raise a cap already set.
   • Batch paths are observed with a fixed label: /v1/batch or /v1/batch/{id}. A job id must never become a label value.
3. Submit (POST /v1/batch), checked in this order:
   • The body is {"targets": [{"url", "mode", "question", "pages", "grep", "jq", "context", "recover"}, …]}, each target flat with /v1/fetch's field names. Bad JSON, a missing or empty list, an empty url, or a mode other than empty, clean, raw or digest → 400 invalid batch: ….
   • s.Guard.PreCheck(url) for every target. The first failure → 422 with the message blocked: target <idx>: <reason>, and no job is created.
   • s.Batch.Submit(r.Context(), c, s.callerClass(c), targets):
     • ErrTooManyTargets → 422 too many targets (max 32) (the configured cap);
     • ErrCallerBusy → 429 busy: a batch of yours is still running, Retry-After: 30;
     • ErrBusy → 429 batch jobs busy, Retry-After: 60;
     • ErrNoTargets → 400.
   • 202 {"success": true, "job_id", "status", "targets": n, "reused"}. status is "queued" for a new job, and the job's current status for a reused one.
4. Ownership: a job whose Caller is not c.Name, or an unknown id, is 404 not found for every GET and DELETE. It is never 403: a caller cannot probe for another caller's jobs.
5. Status (GET /v1/batch/{id}): after the ownership check, s.Batch.Touch(id), then 200 {"job_id", "status", "error" (only when set), "targets_total", "targets_done", "created_at" (RFC 3339, UTC), "targets": [{"idx", "url", "status", "error"?, "hint"?}]}.
   • This is a summary: no results, because 32 inline results can be 6 MB, too much for one agent tool call.
   • With ?results=1, each done target also carries "result": the envelope, i.e. Meta plus "content".
6. One result (GET /v1/batch/{id}/{idx}): after ownership and Touch, 200 {"job_id", "idx", "url", "status", "result"?, "error"?, "hint"?}. An idx that is not a number in 0..targets_total-1 → 404. Cancel (DELETE): after ownership, a terminal job → 409 job already finished; otherwise Cancel → 200 {"success": true, "status": "cancelled"}.
7. Logging: each batch handler logs one line: batch with caller, job_id, targets and status (or reused). Never a target URL or hostname, for any caller, and never the idem_key. The per-target fetch line inside fetchTarget is unchanged: it is redacted for restricted callers exactly as for /v1/fetch.

Facts

• newBatchServer, pageServer and fakeDNS come from batchrun_test.go (task 05). Targets are local httptest servers, some held on a channel, or internal.test (resolves to 10.0.0.5).
• TestBatchBodyCap posts 32 targets with questions, about 66 KiB: over maxReqBody, under batchMaxBody.

Steps

• 1. git switch p6; copy the given test. 2. See it fail.
• 3. The handlers. 4. Routes and ServeHTTP.
• 5. gofmt -w; go test -race -count=3 ./internal/api/ → ok. 6. make gate.
• 7. Row p6/06-api-endpoints; commit.

git add internal/api docs/implementer-log.md
git commit -F .state/commit-msg.txt

Done when

• The given test and every existing api test pass under -race -count=3; make gate ok; the given file is byte-identical.

Stop and report if

• Any given test needs a URL that is not local, or a change to an existing test.