Commit Graph
7 Commits
Author SHA1 Message Date
kyle 385529da39 metrics: phase 3 straylight/rift scrapes and hyperborea units
Review, no switch. node-laptop for talos/titan is excluded from
FleetTargetDown. llama-straylight scrape uses params.model for
ornith-1.5-35b-a3b (router 400 without it; will load if cold).
2026-09-24 00:42:06 -07:00
kyle b861c6ee9f metrics: drop llama-dixie-vision scrape
Router /metrics cannot be up without loading a model.
2026-09-24 00:29:24 -07:00
kyle ed5e1a1e0e metrics: HonchoDeriverBacklog and include honcho in target-down 2026-09-24 00:21:36 -07:00
kyle 5d8e8c4335 metrics: scrape dixie helper /metrics with model param
Router-mode llama-server returns 400 without it.
2026-09-23 23:57:36 -07:00
kyle b1d5aa7e64 metrics: drop alertmanager channel override
Webhook is already bound to ~alerts; Mattermost's channel
override wants the name, not ~alerts.
2026-09-23 23:44:24 -07:00
kyle ec74e0b69c metrics: phase 1 follow-up (review, no switch)
Honcho workspace_name keep, node_exporter restart metrics,
drop min_over_time on rule 1, Watchdog heartbeat, pin tether
to b267e2f so labelled series exist and rule 2 can fire.
2026-09-23 23:43:11 -07:00
kyle 4958493485 metrics: phase 1 orion+dixie (review, no switch)
VictoriaMetrics on 127.0.0.1:8428. LoadCredential for the Mattermost
webhook and the sift scrape token. Honcho scrapes keep a closed metric
name set. Meta-alerts for target-down, VM size, orion disk. Dixie gets
node_exporter, nvidia_gpu_exporter, and llama-server --metrics.

Tether labelled series still wait on titan's fix. Rule 2 is written
against reason=client_disconnect so it stays silent until then.
2026-09-23 23:22:38 -07:00