crossbar/docs/plans/v2.1/02-props-loaded-only.md

v2.1 task 02: router mode — loaded means loaded, context is per model

Branch: v2.1 (git switch v2.1; git status --short must be empty, otherwise stop) Commit subject: Learn per-model context from /props?model=; only status "loaded" is loaded

Goal

crossbar's real upstreams are llama-server routers, not single servers, and v2's poller was written against the single-server shape. On a router: /v1/models lists every configured model with a status.value ("loaded", "unloaded", "loading"); the plain /props answers as the router itself ("role":"router", n_ctx 0); and /props?model=X answers for X's child server — and loads X if it is not loaded, which a health poll must never cause. After this task the poller treats only "loaded" models as loaded, asks /props?model=X only for those, keeps the answers per model, and the guard and the hosts view use the per-model figures. A plain single server keeps working exactly as in v2.

Context

Verified on straylight's router on 2026-09-25: /v1/models entries carry "status":{"value":"unloaded",...} for seven of eight models; plain /props returns {"role":"router","model_alias":"llama-server","default_generation_settings":{"n_ctx":0}}; /props?model=ornith-1.5-35b-a3b (loaded) returns n_ctx 262144 and total_slots 4. With v2's code every listed model counts as loaded and the guard learns nothing, so the "Context size has been exceeded" failure crossbar exists to prevent still happens on a router.

Files

• Copy: internal/health/props_router_test.go, internal/proxy/ctxguard_router_test.go, internal/admin/admin_models_test.go
• Modify: internal/health/health.go (a new internal/health/props.go is allowed for the 400-line limit), internal/proxy/ctxguard.go, internal/admin/admin.go, docs/implementer-log.md

Interfaces

package health

// ModelCtx is what /props?model=X taught us about one loaded model.
type ModelCtx struct {
    NCtx  int `json:"n_ctx"`
    Slots int `json:"slots"`
}

type Status struct {
    // ... as v2 ...
    Models map[string]ModelCtx `json:"models"` // per loaded model; never nil after a poll; copied by Get/All
}

// PerSlotCtxFor is the per-slot context for one model on this host: Models[model] when present
// (NCtx/Slots, 0 when either is 0); else, when model is in Loaded, the host-level PerSlotCtx();
// else 0 ("unknown" / not resident).
func (s Status) PerSlotCtxFor(model string) int

Poller rules:

1. /v1/models: an entry is loaded when it has no status or status.value == "loaded"; any other value ("unloaded", "loading", …) is not loaded and does not appear in Loaded.
2. Plain /props: when the body has "role":"router" the host-level NCtx/Slots stay 0 whatever else it says; otherwise as v2 (best effort, never a failure).
3. For each model in Loaded, GET /props?model=<url.QueryEscape(id)>, decoded like the plain one, into Models[id]. A failed or malformed answer leaves that id absent and the host healthy. Never ask for a model that is not in Loaded.
4. Models is a fresh non-nil map on every successful poll; MarkDown leaves it as last seen.

Proxy (ctxguard.go): every use of PerSlotCtx() becomes PerSlotCtxFor(model) — the leased host's figure, the candidates a prompt may move to, the wake path's "cannot serve it no matter how it wakes" check, and largestSlotCtx, which now takes the model and so only counts hosts that have it loaded. Admin (admin.go): HostView gains Models map[string]health.ModelCtx (JSON models, an empty object never null).

Steps

• 1. git switch v2.1; copy the three given tests.
• 2. See them fail: go test -race -count=1 ./internal/health/ ./internal/proxy/ ./internal/admin/ — the new tests fail to compile until the names exist, then fail on behaviour.
• 3. health first (rules 1–4, ModelCtx, PerSlotCtxFor); go test -race -count=1 ./internal/health/ → ok.
• 4. ctxguard.go, then admin.go; each package ok. gofmt -w.
• 5. go test -race -count=1 ./... → all ok; make smoke → smoke: ok (the smoke's fake is a single server; nothing there changes).
• 6. make gate. 7. Row v2.1/02-props-loaded-only; commit.

git add internal/health internal/proxy internal/admin docs/implementer-log.md
git commit

Done when

• All three given tests pass; every v2 health/guard/admin test still passes; gate and smoke ok; given files byte-identical; no file over 400 lines.

Stop and report if

• A v2 given test (props_test.go, ctxguard_test.go, admin_test.go, health_test.go) needs changing to pass: quote it — that is the owner's test, not yours to edit.