crossbar/docs/plans/v0/02-health.md

v0 task 02: the health poller

Branch: v0 (run git switch v0; git status --short must be empty, otherwise stop) Commit subject: Add the health poller

Goal

Write internal/health: a table that knows, for every host, whether it answered its last poll, which models it has loaded, and when it last answered. The proxy (task 03) reads this table to choose a host and tells it when a request to a host fails.

Context

A llama-server router answers GET /health with 200 when it can serve (503 while loading), and GET /v1/models with {"object":"list","data":[{"id":"<model>", …}, …]} listing the models it has resident. A host that failed must answer two polls in a row before it is trusted again (RecoveryPolls), because a router that is loading a model flaps. A host that has never failed is trusted after its first good poll. Never call /slots: probing an unloaded model makes the router load it.

Files

• Copy: internal/health/health_test.go
• Create: internal/health/health.go
• Modify: docs/implementer-log.md

Interfaces

Produces, in internal/health/health.go, package health:

const RecoveryPolls = 2              // good polls in a row a host needs after a failure
const MaxModelsBody = 1 << 20        // bound on what we read from /v1/models

type Status struct {
    Healthy     bool      `json:"healthy"`
    Loaded      []string  `json:"loaded"`      // sorted, unique model ids from the last good poll
    LastOK      time.Time `json:"last_ok"`     // zero if never
    LastErr     string    `json:"last_err"`    // "" after a good poll
    Consecutive int       `json:"consecutive"` // good polls in a row
}

type Table struct { /* private: a mutex, name -> base URL, name -> status + "ever failed" flag, client, interval */ }

func New(hosts map[string]string, interval time.Duration, client *http.Client) *Table
func (t *Table) Run(ctx context.Context)          // PollOnce now, then every interval; returns when ctx is done
func (t *Table) PollOnce(ctx context.Context)     // polls every host concurrently, returns when all are done
func (t *Table) Get(name string) (Status, bool)   // a copy; ok == false for an unknown name
func (t *Table) All() map[string]Status           // copies
func (t *Table) MarkDown(name, reason string)     // a failure seen by the proxy

Rules the tests check:

1. New: hosts maps a host name to its base URL (no trailing slash; the config already trimmed it). A nil client becomes &http.Client{Timeout: 5 * time.Second}. Every host starts with Healthy false, Loaded an empty slice, not nil, Consecutive 0.
2. One poll of one host is two requests with ctx: GET <base>/health must return 200, else the poll fails with LastErr starting health: (for example health: HTTP 503, or the client error); then GET <base>/v1/models must return 200 and decode as {"data":[{"id":"…"}]}, else the poll fails with LastErr starting models: . Read at most MaxModelsBody bytes of either body. Loaded is the ids, empty ids dropped, duplicates dropped, sorted.
3. Success: Consecutive++, LastOK = time.Now(), LastErr = "", Loaded replaced, and Healthy = (never failed) || Consecutive >= RecoveryPolls.
4. Failure, from a poll or from MarkDown: remember that the host has failed, Healthy = false, Consecutive = 0, LastErr = reason. Loaded is left as last seen. MarkDown uses reason = "marked down: " + reason. Unknown names are ignored, never a panic.
5. A cancelled context is not a failure. If a request fails and ctx.Err() != nil, record nothing: we were told to stop, that says nothing about the host.
6. Run polls once immediately, then on a time.Ticker every interval, and returns when ctx is done (stop the ticker). PollOnce polls all hosts concurrently (one goroutine each, a sync.WaitGroup) and returns when all have finished.
7. Get and All return copies: changing Loaded on what they return must not change the table (copy the slice). All methods are safe to call from several goroutines: one mutex around the map, never held while a request is in flight.

Steps

• 1. Copy.

git switch v0
cp docs/plans/v0/_files/internal/health/health_test.go internal/health/

Read the test. newFake is the router stand-in; TestFailureThenRecoveryNeedsTwoPolls is rule 3 and 4 in one story.

• 2. See the test fail. go test ./internal/health/. Expected: it does not compile.
• 3. Write internal/health/health.go. gofmt -w internal/health/.
• 4. See the test pass. go test -race -count=1 ./internal/health/. Expected: ok. TestRunPollsOnStart polls a fake every 20 ms; if it fails, check rules 5 and 6.
• 5. Run the gate. make gate. Expected last line: gate: ok.
• 6. Log and commit. Row v0/02-health.

git add internal/health docs/implementer-log.md
git commit

Done when

• go test -race -count=1 ./internal/health/ is ok; make gate prints gate: ok.
• cmp internal/health/health_test.go docs/plans/v0/_files/internal/health/health_test.go prints nothing.

Stop and report if

• The race detector reports a race you cannot remove with the one-mutex design above.