P6 plan: asynchronous batch fetch endpoint — phase spec, task plan, given tests

Co-Authored-By: DeepSeek V4 Pro <noreply@deepseek.com>
2026-09-26 12:09:38 -07:00
parent b1da18d49e
commit 63e70e19fc
12 changed files with 1564 additions and 0 deletions
@@ -0,0 +1,70 @@
+# 05-cli-docs — `sift batch`, skill, README, config
+
+Expose the batch endpoint from the CLI and document it for agents. No Hermes plugin — the CLI is
+the agent surface, same rationale as `sift fetch`/`sift search`.
+
+## CLI
+
+Add a `sift batch` subcommand with three verbs. It reads `url` and `key_file` from `client.toml`
+exactly as `fetch`/`search` do, and never puts a key on the command line.
+
+```
+sift batch submit <targets.json>      # '-' reads stdin; prints the job_id (and reused) as JSON
+sift batch status <job_id>            # prints status + per-target results as JSON
+sift batch cancel <job_id>            # prints the cancelled status as JSON
+```
+
+`submit` takes the same JSON as `POST /v1/batch` (a `{"targets":[…]}` object) and prints
+`{"job_id": …, "reused": …}`. `status` prints the full `GET /v1/batch/{id}` body. All three
+propagate the server's error envelope and exit non-zero on a non-2xx.
+
+The dispatch/validation is a pure function pinned by the given test:
+
+```go
+func parseBatchArgs(args []string) (verb, arg string, err error)
+```
+
+`verb` is `submit`/`status`/`cancel`; `arg` is the targets file (submit, `-` for stdin) or the
+`job_id` (status/cancel). No verb, a missing arg, an extra arg, or an unknown verb is an error.
+
+## Config
+
+Add a `[batch]` block to `internal/config` (`File`), with:
+
+```toml
+[batch]
+targets_per_job = 32
+jobs_per_caller = 1
+jobs_total      = 8
+ttl             = "24h"
+max_bytes       = "64MiB"
+```
+
+The runner and API read these instead of hard-coding; the defaults match the settled caps. The
+body-cap knob for `/v1/batch` (256 KiB) is a constant, not config.
+
+## Docs
+
+- `README.md`: add a short "Batch" section (the three endpoints, the serial/background model, caps,
+  idempotency, lease/TTL semantics, restricted-caller redaction, and the `sift_batch_*` metrics).
+- `skill/SKILL.md`: add a batch section telling agents to use `sift batch submit` for a list of
+  targets rather than a loop of `sift fetch`, and to `sift batch status` to collect.
+- `dist/sift.toml.example`: add the `[batch]` block with its defaults and a comment.
+
+## Rules
+
+1. The CLI reuses the existing client plumbing (`client.toml`, bearer header, JSON envelope
+   handling) — no new HTTP client.
+2. `submit` validates its input locally (a non-empty `targets` array) before calling the server.
+3. `status`/`cancel` require exactly one `job_id` argument.
+4. Keep each verb a small function; do not grow `main.go` — add `cmd/sift/batchcmd.go`.
+
+## Given
+
+Copy `_files/cmd_sift/batch_test.go` → `cmd/sift/batch_test.go`. It pins `parseBatchArgs` — the
+valid verb+arg pairs (including `-` as a stdin target file) and the error cases (no verb, missing
+arg, extra arg, unknown verb).
+
+## Gate
+
+`make gate` green; `go test ./cmd/sift/` green under `-race`; `go test ./internal/config/` green.