PLAN rev 3: TOML config

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-09-25 00:44:38 -07:00
co-authored by Claude Fable 5.1
parent 9f41ce7935
commit fe51c20c10
+42 -30
View File
@@ -1,6 +1,6 @@
# crossbar — an affinity router for the fleet's llama-servers
**Status:** plan, 2026-09-25 (rev 2: SQLite state + accounting, Kyle). Nothing built. Go.
**Status:** plan, 2026-09-25 (rev 2: SQLite state + accounting; rev 3: TOML config — Kyle). Nothing built. Go.
**Owner:** Kyle. Drafted by claude from the 2026-09-25 discussion (Inference Infrastructure).
## 1. Problem
@@ -145,35 +145,47 @@ exposes the rollups; a nightly job prunes `requests` older than `retention`
Body contents are never stored — only counts and timings.
## 8. Config (`crossbar.yaml`)
## 8. Config (`crossbar.toml`)
```yaml
listen: "100.x.y.z:7777" # tailnet address only; never 0.0.0.0
db: /var/lib/crossbar/crossbar.db
retention: 180d
poll_interval: 60s
lease_idle: 30m
queue_max: 8
hosts:
straylight:
base_url: http://straylight.<tailnet>:11434
weight: 1.0
models: { ornith-1.5-35b-a3b: {parallel: 4}, ornith-1.5-9b-uncensored: {parallel: 6} }
titan:
base_url: http://titan.<tailnet>:8081
weight: 2.0 # ~2× straylight decode
models: { ornith-1.5-35b-a3b: {parallel: 4}, laguna-s-2.1: {parallel: 2} }
dixie:
base_url: http://dixie.<tailnet>:11434
weight: 0.8
models: { ornith-1.5-9b-uncensored: {parallel: 6} }
routes: # optional defaults / pins
opencode-a: { default_model: ornith-1.5-35b-a3b }
opencode-b: { default_model: ornith-1.5-35b-a3b }
paper: { default_model: qwen3.8-27b-uncensored, pin: titan }
hermes-straylight: {}
hermes-talos: {}
hermes-titan: { pin: titan } # titan's models are the titan agent's first
TOML (`github.com/BurntSushi/toml`): no indentation semantics, no implicit
type coercion, and it is what Kyle's Rust projects already use. A NixOS module
can generate it with `pkgs.formats.toml`.
```toml
listen = "100.x.y.z:7777" # tailnet address only; never 0.0.0.0
db = "/var/lib/crossbar/crossbar.db"
poll_interval = "60s"
lease_idle = "30m"
queue_max = 8
retention = "180d"
[hosts.straylight]
base_url = "http://straylight.<tailnet>:11434"
weight = 1.0
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "ornith-1.5-9b-uncensored" = { parallel = 6 } }
[hosts.titan]
base_url = "http://titan.<tailnet>:8081"
weight = 2.0 # ~2x straylight decode
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "laguna-s-2.1" = { parallel = 2 } }
[hosts.dixie]
base_url = "http://dixie.<tailnet>:11434"
weight = 0.8
models = { "ornith-1.5-9b-uncensored" = { parallel = 6 } }
# optional per-route defaults / pins
[routes.opencode-a]
default_model = "ornith-1.5-35b-a3b"
[routes.opencode-b]
default_model = "ornith-1.5-35b-a3b"
[routes.paper]
default_model = "qwen3.8-27b-uncensored"
pin = "titan"
[routes.hermes-straylight]
[routes.hermes-talos]
[routes.hermes-titan]
pin = "titan" # titan's models are the titan agent's first
```
Client side, no code changes:
@@ -201,7 +213,7 @@ Client side, no code changes:
## 10. Milestones
- **v0 (a day):** config, static routes → host mapping, reverse proxy with
- **v0 (a day):** TOML config, static routes → host mapping, reverse proxy with
streaming, `/health` + `/models` poller, passive health, `/_crossbar/hosts`.
Replaces the hand-maintained provider lists. No leases yet: each route has
a fixed host list in preference order; first healthy wins.