v2.3 plan: clients that manage their own slots (control calls, route affinity, queue = false, route listeners, llama-server ctx error); given tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 17:36:59 -07:00
parent 4c6415819e
commit 3518e84dd7
16 changed files with 1655 additions and 0 deletions
@@ -0,0 +1,56 @@
+# v2.3 task 04: llama-server's error shape for a context refusal; README
+
+**Branch:** `v2.3` (`git switch v2.3`; `git status --short` must be empty, otherwise stop)
+**Commit subject:** `Context refusal in llama-server's exceed_context_size_error shape; README for v2.3`
+
+## Goal
+
+When no host can fit a prompt, crossbar answers `400 {"error":"prompt too large","estimate":N,"max":M}`.
+A client that already handles llama-server's own overflow error (Boxmaker keys on `error.type`
+and reads only the first 4 KiB) does not recognise it. After this task the body is the server's
+shape, so the client handles crossbar's refusal like the server's:
+
+```json
+{"error":{"code":400,"type":"exceed_context_size_error","message":"prompt too large","n_prompt_tokens":N,"n_ctx":M}}
+```
+
+`N` is the estimate and `M` the largest per-slot context on the route, as before.
+
+## Files
+
+- Copy: the **replacement** `internal/proxy/ctxguard_test.go` (was v2's; only the 400-body
+  assertions changed; the v2.3 copy is now protected)
+- Modify: `internal/proxy/ctxguard.go` (`refuseCtx`), `README.md`, `docs/implementer-log.md`
+
+## Rules
+
+1. The body is exactly one JSON object whose only top-level key is `"error"`, so it starts with
+   `{"error":`; `Content-Type: application/json`; status 400. The accounting row is unchanged
+   (status 400, `Err` "prompt too large"). Every other crossbar error keeps its current
+   `{"error":"<text>"}` shape.
+2. `README.md`:
+   - The context-guard section: the new body.
+   - A new section **"Clients that manage their own slots"** covering: control calls (which
+     requests, and that they follow the lease but take no slot, skip the guard and write no
+     row); `/slots` and `/tokenize` are proxied, `/slots/<id>` actions are not; a GET's model
+     comes from `?model=`; the route keys `affinity`, `queue` and `listen` with the
+     `boxmaker-a` example from `example.toml`; that `listen` is refused on templates and must
+     not be the main address; that the admin API is not served on a route listener.
+   - The "hosts view"/config reference tables, if they list route keys, gain the three keys.
+
+## Steps
+
+- [ ] **1.** `git switch v2.3`; copy the replacement test. **2. See it fail** (old body).
+- [ ] **3.** `refuseCtx`. **4.** README.
+- [ ] **5.** `gofmt -w`; `make gate`; `make smoke` (check 3 still finds `"prompt too large"`).
+- [ ] **6.** Row `v2.3/04-ctx-error-docs`; commit.
+
+```sh
+git add internal/proxy README.md docs/implementer-log.md
+git commit
+```
+
+## Done when
+
+- All tests pass; gate and smoke ok; given files byte-identical; README describes what v2.3
+  does and nothing it does not.