PLAN: future work — asynchronous batch fetch endpoint (Kyle, after P5c)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-26 11:03:43 -07:00
co-authored by Claude Opus 5.5
parent 49d96e197b
commit b1da18d49e
+24
View File
@@ -310,3 +310,27 @@ harness); commit + push to `git.wntrmute.dev:kyle/sift`; NixOS changes go
through `~/src/nixos` committed master and `rebuild-nixos` (fetch before editing — clones go through `~/src/nixos` committed master and `rebuild-nixos` (fetch before editing — clones go
stale); state directories are never deleted; secrets never printed; one phase at a time with stale); state directories are never deleted; secrets never printed; one phase at a time with
its acceptance evidence posted in `~agent`. its acceptance evidence posted in `~agent`.
## 13. Future work
### Batch fetch endpoint (Kyle, 2026-09-26; plan after P5c)
A deliberate, **slow**, asynchronous endpoint for an agent that has collected a series of
targets and wants them fetched without holding a request open per target. It builds on the
batching infrastructure that will exist by the time P5c is deployed.
- **Submit:** `POST /v1/batch` with a list of targets (URLs, and presumably the same per-target
options as `/v1/fetch`: `mode`, `question`, `pages`, `grep`, `jq`). Answers **202** with a job
ID when accepted, **400** for a malformed request, **422** when a target is refused up front
(for example the guard blocks it, or the list is over the cap).
- **Poll and collect:** the caller checks in on the job ID later (`GET /v1/batch/{id}`: status and
per-target progress) and fetches the results when they are done.
- Questions for the design, when it is planned:
- caps: targets per job, jobs per caller, total bytes;
- per-target results and errors (one bad URL does not fail the job);
- how long results are kept, and where (the SQLite jobs table the P5 pool discussion
anticipated, under `/var/lib/sift`);
- scheduling behind interactive requests (the same slots and gates, lower priority);
- restricted callers: their jobs, results and logs partitioned like everything else;
- cancellation and idempotency (resubmitting the same list);
- whether Hermes needs a tool or skill change to use it.