diff --git a/docs/PLAN.md b/docs/PLAN.md index b2b03c6..a2c9f8e 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -310,3 +310,27 @@ harness); commit + push to `git.wntrmute.dev:kyle/sift`; NixOS changes go through `~/src/nixos` committed master and `rebuild-nixos` (fetch before editing — clones go stale); state directories are never deleted; secrets never printed; one phase at a time with its acceptance evidence posted in `~agent`. + +## 13. Future work + +### Batch fetch endpoint (Kyle, 2026-09-26; plan after P5c) + +A deliberate, **slow**, asynchronous endpoint for an agent that has collected a series of +targets and wants them fetched without holding a request open per target. It builds on the +batching infrastructure that will exist by the time P5c is deployed. + +- **Submit:** `POST /v1/batch` with a list of targets (URLs, and presumably the same per-target + options as `/v1/fetch`: `mode`, `question`, `pages`, `grep`, `jq`). Answers **202** with a job + ID when accepted, **400** for a malformed request, **422** when a target is refused up front + (for example the guard blocks it, or the list is over the cap). +- **Poll and collect:** the caller checks in on the job ID later (`GET /v1/batch/{id}`: status and + per-target progress) and fetches the results when they are done. +- Questions for the design, when it is planned: + - caps: targets per job, jobs per caller, total bytes; + - per-target results and errors (one bad URL does not fail the job); + - how long results are kept, and where (the SQLite jobs table the P5 pool discussion + anticipated, under `/var/lib/sift`); + - scheduling behind interactive requests (the same slots and gates, lower priority); + - restricted callers: their jobs, results and logs partitioned like everything else; + - cancellation and idempotency (resubmitting the same list); + - whether Hermes needs a tool or skill change to use it.