crossbar/docs/plans/v2/02-ctxguard.md

v2 task 02: the context-size guard

Branch: v2 (run git switch v2; git status --short must be empty, otherwise stop) Commit subject: Proxy: move or refuse prompts that do not fit the leased host's context

Goal

The single worst failure a client sees today is the upstream's "Context size has been exceeded" after a long wait. With PerSlotCtx known (task 01), crossbar can estimate a prompt's size from its body and act before forwarding: move the conversation to a host where it fits, or answer 400 with the estimate and the largest slot available. PLAN.md §4b.

Files

• Copy: internal/proxy/ctxguard_test.go
• Modify: internal/proxy/proxy.go and/or forward.go (new file ctxguard.go if that keeps files under 400 lines), internal/lease/lease.go (one addition, below), docs/implementer-log.md

Interfaces

// internal/proxy
const CtxHeader = "X-Crossbar-Ctx"      // set only when the guard moved a conversation: "moved:<from>>><to>" e.g. "moved:small>big"

// internal/lease — one addition, the only change allowed there:
// Move re-leases k onto host (deleting any existing lease for k), records a LeaseEvent with
// Reason "ctx", FromHost the previous host ("" if none), ToHost host. Returns an error only from
// the persister.
func (t *Table) Move(k Key, host string, now time.Time) error

Rules the tests check (ServeHTTP, between acquiring the lease and taking a slot):

1. estimate := int(float64(len(body)) / 4 * 1.2) tokens (body = the bytes already peeked; GET/HEAD → 0).
2. limit := PerSlotCtx of the leased host's health.Status. If limit == 0 (unknown) or estimate <= limit: no action, no header.
3. Otherwise find, among the route's candidate hosts (in order), the healthy, non-draining hosts whose PerSlotCtx() >= estimate — prefer one that lists the model as loaded, else one that can serve it (cfg.Serves). If one exists: leases.Move(key, host, now), set CtxHeader to moved:<old>><new>, and continue with the new host (this request and the following turns: the lease moved).
4. If none exists: 400 {"error":"prompt too large","estimate":E,"max":M} where M is the largest PerSlotCtx() among the route's healthy hosts (0 if all unknown — but then rule 2 already let the request through). Record an accounting row with status 400 and Err: "prompt too large"; do not mark anything down; do not forward.
5. The estimate is never logged with the body; the log line gains ctx_est=E only.

Add ReasonCtx = "ctx" to internal/store constants (one-line change, allowed).

Steps

• 1. git switch v2; cp docs/plans/v2/_files/internal/proxy/ctxguard_test.go internal/proxy/.
• 2. See it fail (compile: CtxHeader). 3. Write the code. gofmt -w internal/.
• 4. go test -race -count=2 ./internal/proxy/ ./internal/lease/ → ok.
• 5. make gate → gate: ok. 6. Row v2/02-ctxguard; commit.

git add internal/proxy internal/lease internal/store docs/implementer-log.md
git commit

Done when

• Tests pass with -race -count=2; gate ok; the copied test is byte-identical.

Stop and report if

• TestStickyLeaseSurvivesGrowthUntilItDoesNotFit fails on the second request after the move (the lease did not actually move): quote the lease table.