126 lines
7.1 KiB
Markdown
126 lines
7.1 KiB
Markdown
# v1 task 06: the admin handler (v1)
|
|
|
|
**Branch:** `v1` (run `git switch v1`; `git status --short` must be empty, otherwise stop)
|
|
**Commit subject:** `Admin: leases, pin, release, drain, usage, metrics`
|
|
|
|
## Goal
|
|
|
|
Operators see and steer the system: the hosts view gains slots and drain state, the routes view
|
|
shows leases and pins, `POST` endpoints pin/release a route and drain a host, `/usage` answers
|
|
the accounting questions, `/metrics` exposes them to Prometheus. `PLAN.md` §7, §7a. The
|
|
`cmd/crossbar` wiring is the next task (07), not this one.
|
|
|
|
**State of the tree when this task starts:** an earlier session already added
|
|
`store.StatusCounts` (`internal/store`) and `lease.Table.Candidates` (`internal/lease`), copied
|
|
`example.toml` and `admin_test.go`, and left a broken draft of `internal/admin/admin.go`
|
|
(duplicate `Handler` declarations). Those files are uncommitted in the working tree. Keep the
|
|
store and lease additions (they pass their tests); treat `admin.go` as scratch you may rewrite
|
|
from a blank file. This task commits all of them.
|
|
|
|
## Files
|
|
|
|
- Already copied (verify with `cmp`, never edit): `internal/admin/admin_test.go`, `example.toml`
|
|
- Modify: `internal/admin/admin.go` (split if over 400 lines), `cmd/crossbar/main.go` (one call, see step 5), `docs/implementer-log.md`
|
|
- Already modified, commit as they are after their tests pass: `internal/store/store.go`, `internal/store/schema.go`, `internal/lease/lease.go`
|
|
|
|
## Interfaces
|
|
|
|
`internal/lease` gains one method — the only change to that package allowed in this task:
|
|
|
|
```go
|
|
// Candidates records hosts as seen for route (idempotent), so Pin can accept a host the route
|
|
// is configured for before any request has used it. cmd/crossbar calls it for every route at
|
|
// start; the admin handler calls it before Pin.
|
|
func (t *Table) Candidates(route string, hosts []string)
|
|
```
|
|
|
|
`internal/admin`, package `admin`:
|
|
|
|
```go
|
|
type Hosts interface { All() map[string]health.Status }
|
|
type Drainer interface { Draining(name string) bool; SetDraining(name string, on bool) } // *proxy.Hosts satisfies it
|
|
|
|
type HostView struct {
|
|
Healthy bool `json:"healthy"`
|
|
Loaded []string `json:"loaded"` // never null
|
|
LastOK string `json:"last_ok"` // RFC 3339 UTC or ""
|
|
LastErr string `json:"last_err"`
|
|
FreeSlots int `json:"free_slots"` // lim.FreeSlots(host)
|
|
InFlight int `json:"in_flight"` // sum over the host's configured models
|
|
Queued int `json:"queued"` // same
|
|
Draining bool `json:"draining"`
|
|
}
|
|
type LeaseView struct { FP, Model, Host, State, Created, LastUsed string } // json tags: fp, model, host, state, created, last_used (RFC 3339 UTC)
|
|
type RouteView struct {
|
|
Hosts []string `json:"hosts"`
|
|
DefaultModel string `json:"default_model"`
|
|
Pinned string `json:"pinned"` // "" when not pinned
|
|
Leases []LeaseView `json:"leases"` // never null
|
|
}
|
|
func Handler(cfg *config.Config, h Hosts, lt *lease.Table, lim *limiter.Limiter, st *store.Store, d Drainer) http.Handler
|
|
```
|
|
|
|
Endpoints (all JSON unless said; errors `{"error":"…"}`; wrong method → 405 with `Allow`):
|
|
|
|
- `GET /_crossbar/hosts` → `map[string]HostView`.
|
|
- `GET /_crossbar/routes` → `map[string]RouteView` from config + `lt.Snapshot()` + `lt.Pinned`.
|
|
- `POST /_crossbar/routes/{route}` body `{"host":"…","pin":true}`: first check `host` is one of
|
|
`cfg.Routes[route].Hosts` (else **404**), then `lt.Candidates(route, cfg.Routes[route].Hosts)` so
|
|
the table knows them even if no request has used the route yet, then `lt.Pin(route, host, now)`;
|
|
`{"release":true}` → `lt.Release(route)` **and** `lt.Unpin(route)`. Unknown route → 404;
|
|
`lt.Pin` returning `ErrUnknownHost` → 404; not JSON, neither form, or both forms at once,
|
|
or `pin` without `host` → 400. Answer `{"ok":true}` (plus `"released": n` for a release).
|
|
`GET` on this path → 405.
|
|
- `POST /_crossbar/hosts/{host}` body `{"drain":true|false}` → `d.SetDraining`; unknown host
|
|
(not in config) → 404; bad body → 400. `{"ok":true}`.
|
|
- `GET /_crossbar/usage?since=…&by=route|model|host` → `[]store.UsageRow` (JSON array; sorted by
|
|
key). `by` defaults to `route`; `since` is either RFC 3339 or a duration like `24h`/`7d`
|
|
(meaning `now - d`); absent = all time; anything else → 400. With `Accept: text/plain`, a
|
|
fixed-width table with a header line containing `key requests errors busy_ms queued_ms
|
|
prompt cached completion cache_hit` and one line per row (`cache_hit` as `0.80`).
|
|
- `GET /_crossbar/metrics` → `text/plain; version=0.0.4`, computed on request. Request counters
|
|
need (route, host, status), which `Usage` (one key) cannot give, so add **one** method to
|
|
`internal/store` — the only change to that package allowed in this task:
|
|
```go
|
|
type StatusCount struct { Route, Host string; Status int; Count int64 }
|
|
func (s *Store) StatusCounts(since time.Time) ([]StatusCount, error) // from `requests` only; rolled-up days are not in it, say so in a comment
|
|
```
|
|
Then emit, in this order:
|
|
```
|
|
# TYPE crossbar_requests_total counter
|
|
crossbar_requests_total{route="…",host="…",status="…"} N (one line per StatusCount)
|
|
# TYPE crossbar_prompt_tokens_total counter
|
|
crossbar_prompt_tokens_total{route="…"} N (Usage(zero, ByRoute))
|
|
# TYPE crossbar_cached_tokens_total counter
|
|
# TYPE crossbar_completion_tokens_total counter
|
|
# TYPE crossbar_queue_wait_ms_total counter crossbar_queue_wait_ms_total{route="…"} N
|
|
# TYPE crossbar_host_healthy gauge crossbar_host_healthy{host="…"} 0|1
|
|
# TYPE crossbar_host_free_slots gauge
|
|
# TYPE crossbar_host_in_flight gauge
|
|
# TYPE crossbar_host_queued gauge
|
|
```
|
|
Label values escaped (`"` and `\`), lines sorted, no trailing spaces.
|
|
|
|
## Steps
|
|
|
|
- [ ] **1. Check the tree.** `git switch v1`; `git status --short` shows the modified store, lease,
|
|
admin and example files listed above. `cmp internal/admin/admin_test.go docs/plans/v1/_files/internal/admin/admin_test.go`
|
|
and `cmp example.toml docs/plans/v1/_files/example.toml` print nothing. If they do not, copy the given files again.
|
|
- [ ] **2. Confirm the inherited pieces pass.** `go test -race -count=1 ./internal/store/ ./internal/lease/`. Expected: both `ok`.
|
|
- [ ] **3. Write `internal/admin/admin.go`** (delete the draft first if it is easier). `gofmt -w internal/admin/`.
|
|
- [ ] **4. See the test pass.** `go test -race -count=1 ./internal/admin/`. Expected: `ok`.
|
|
- [ ] **5. Make the module build.** The new `admin.Handler` signature breaks the one call in
|
|
`cmd/crossbar/main.go`; change that call to `admin.Handler(cfg, table, nil, nil, nil, nil)` and
|
|
nothing else in that file (task 07 wires the real values). Then `make gate`. Expected last line:
|
|
`gate: ok`.
|
|
- [ ] **6. Log and commit.** Row `v1/06-admin`. Deviations: say that the store and lease additions came from the earlier session.
|
|
|
|
```sh
|
|
git add internal/admin internal/store internal/lease cmd/crossbar example.toml docs/implementer-log.md
|
|
git commit
|
|
```
|
|
|
|
## Done when
|
|
|
|
- `go test -race -count=1 ./...` passes; `make gate` prints `gate: ok`; `admin_test.go` and `example.toml` are byte-identical to `_files/`.
|