v0 plan: AGENTS.md, gate, five task files with given tests, run-plan driver
Tests were run against a private reference implementation: gate ok after every task in order, smoke ok (stream spread ~1000 ms). The reference is not in the repository. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,109 @@
|
||||
# v0 task 02: the health poller
|
||||
|
||||
**Branch:** `v0` (run `git switch v0`; `git status --short` must be empty, otherwise stop)
|
||||
**Commit subject:** `Add the health poller`
|
||||
|
||||
## Goal
|
||||
|
||||
Write `internal/health`: a table that knows, for every host, whether it answered its last poll,
|
||||
which models it has loaded, and when it last answered. The proxy (task 03) reads this table to
|
||||
choose a host and tells it when a request to a host fails.
|
||||
|
||||
## Context
|
||||
|
||||
A `llama-server` router answers `GET /health` with 200 when it can serve (503 while loading),
|
||||
and `GET /v1/models` with `{"object":"list","data":[{"id":"<model>", …}, …]}` listing the models
|
||||
it has resident. A host that failed must answer **two** polls in a row before it is trusted again
|
||||
(`RecoveryPolls`), because a router that is loading a model flaps. A host that has never failed is
|
||||
trusted after its first good poll. **Never call `/slots`**: probing an unloaded model makes the
|
||||
router load it.
|
||||
|
||||
## Files
|
||||
|
||||
- Copy: `internal/health/health_test.go`
|
||||
- Create: `internal/health/health.go`
|
||||
- Modify: `docs/implementer-log.md`
|
||||
|
||||
## Interfaces
|
||||
|
||||
Produces, in `internal/health/health.go`, package `health`:
|
||||
|
||||
```go
|
||||
const RecoveryPolls = 2 // good polls in a row a host needs after a failure
|
||||
const MaxModelsBody = 1 << 20 // bound on what we read from /v1/models
|
||||
|
||||
type Status struct {
|
||||
Healthy bool `json:"healthy"`
|
||||
Loaded []string `json:"loaded"` // sorted, unique model ids from the last good poll
|
||||
LastOK time.Time `json:"last_ok"` // zero if never
|
||||
LastErr string `json:"last_err"` // "" after a good poll
|
||||
Consecutive int `json:"consecutive"` // good polls in a row
|
||||
}
|
||||
|
||||
type Table struct { /* private: a mutex, name -> base URL, name -> status + "ever failed" flag, client, interval */ }
|
||||
|
||||
func New(hosts map[string]string, interval time.Duration, client *http.Client) *Table
|
||||
func (t *Table) Run(ctx context.Context) // PollOnce now, then every interval; returns when ctx is done
|
||||
func (t *Table) PollOnce(ctx context.Context) // polls every host concurrently, returns when all are done
|
||||
func (t *Table) Get(name string) (Status, bool) // a copy; ok == false for an unknown name
|
||||
func (t *Table) All() map[string]Status // copies
|
||||
func (t *Table) MarkDown(name, reason string) // a failure seen by the proxy
|
||||
```
|
||||
|
||||
Rules the tests check:
|
||||
|
||||
1. **`New`**: `hosts` maps a host name to its base URL (no trailing slash; the config already
|
||||
trimmed it). A nil `client` becomes `&http.Client{Timeout: 5 * time.Second}`. Every host
|
||||
starts with `Healthy false`, `Loaded` an **empty slice, not nil**, `Consecutive 0`.
|
||||
2. **One poll of one host** is two requests with `ctx`: `GET <base>/health` must return 200,
|
||||
else the poll fails with `LastErr` starting `health: ` (for example `health: HTTP 503`, or the
|
||||
client error); then `GET <base>/v1/models` must return 200 and decode as
|
||||
`{"data":[{"id":"…"}]}`, else the poll fails with `LastErr` starting `models: `. Read at most
|
||||
`MaxModelsBody` bytes of either body. `Loaded` is the ids, empty ids dropped, duplicates
|
||||
dropped, sorted.
|
||||
3. **Success**: `Consecutive++`, `LastOK = time.Now()`, `LastErr = ""`, `Loaded` replaced, and
|
||||
`Healthy = (never failed) || Consecutive >= RecoveryPolls`.
|
||||
4. **Failure**, from a poll or from `MarkDown`: remember that the host has failed, `Healthy =
|
||||
false`, `Consecutive = 0`, `LastErr = reason`. `Loaded` is **left as last seen**.
|
||||
`MarkDown` uses `reason = "marked down: " + reason`. Unknown names are ignored, never a panic.
|
||||
5. **A cancelled context is not a failure.** If a request fails and `ctx.Err() != nil`, record
|
||||
nothing: we were told to stop, that says nothing about the host.
|
||||
6. **`Run`** polls once immediately, then on a `time.Ticker` every `interval`, and returns when
|
||||
`ctx` is done (stop the ticker). **`PollOnce`** polls all hosts concurrently (one goroutine
|
||||
each, a `sync.WaitGroup`) and returns when all have finished.
|
||||
7. **`Get` and `All` return copies**: changing `Loaded` on what they return must not change the
|
||||
table (copy the slice). All methods are safe to call from several goroutines: one mutex around
|
||||
the map, never held while a request is in flight.
|
||||
|
||||
## Steps
|
||||
|
||||
- [ ] **1. Copy.**
|
||||
|
||||
```sh
|
||||
git switch v0
|
||||
cp docs/plans/v0/files/internal/health/health_test.go internal/health/
|
||||
```
|
||||
|
||||
Read the test. `newFake` is the router stand-in; `TestFailureThenRecoveryNeedsTwoPolls` is rule 3
|
||||
and 4 in one story.
|
||||
|
||||
- [ ] **2. See the test fail.** `go test ./internal/health/`. Expected: it does not compile.
|
||||
- [ ] **3. Write `internal/health/health.go`.** `gofmt -w internal/health/`.
|
||||
- [ ] **4. See the test pass.** `go test -race -count=1 ./internal/health/`. Expected: `ok`.
|
||||
`TestRunPollsOnStart` polls a fake every 20 ms; if it fails, check rules 5 and 6.
|
||||
- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`.
|
||||
- [ ] **6. Log and commit.** Row `v0/02-health`.
|
||||
|
||||
```sh
|
||||
git add internal/health docs/implementer-log.md
|
||||
git commit
|
||||
```
|
||||
|
||||
## Done when
|
||||
|
||||
- `go test -race -count=1 ./internal/health/` is `ok`; `make gate` prints `gate: ok`.
|
||||
- `cmp internal/health/health_test.go docs/plans/v0/files/internal/health/health_test.go` prints nothing.
|
||||
|
||||
## Stop and report if
|
||||
|
||||
- The race detector reports a race you cannot remove with the one-mutex design above.
|
||||
Reference in New Issue
Block a user