PLAN rev 3: TOML config

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-09-25 00:44:38 -07:00
co-authored by Claude Fable 5.1
parent 9f41ce7935
commit fe51c20c10
+42 -30
View File
@@ -1,6 +1,6 @@
# crossbar — an affinity router for the fleet's llama-servers # crossbar — an affinity router for the fleet's llama-servers
**Status:** plan, 2026-09-25 (rev 2: SQLite state + accounting, Kyle). Nothing built. Go. **Status:** plan, 2026-09-25 (rev 2: SQLite state + accounting; rev 3: TOML config — Kyle). Nothing built. Go.
**Owner:** Kyle. Drafted by claude from the 2026-09-25 discussion (Inference Infrastructure). **Owner:** Kyle. Drafted by claude from the 2026-09-25 discussion (Inference Infrastructure).
## 1. Problem ## 1. Problem
@@ -145,35 +145,47 @@ exposes the rollups; a nightly job prunes `requests` older than `retention`
Body contents are never stored — only counts and timings. Body contents are never stored — only counts and timings.
## 8. Config (`crossbar.yaml`) ## 8. Config (`crossbar.toml`)
```yaml TOML (`github.com/BurntSushi/toml`): no indentation semantics, no implicit
listen: "100.x.y.z:7777" # tailnet address only; never 0.0.0.0 type coercion, and it is what Kyle's Rust projects already use. A NixOS module
db: /var/lib/crossbar/crossbar.db can generate it with `pkgs.formats.toml`.
retention: 180d
poll_interval: 60s ```toml
lease_idle: 30m listen = "100.x.y.z:7777" # tailnet address only; never 0.0.0.0
queue_max: 8 db = "/var/lib/crossbar/crossbar.db"
hosts: poll_interval = "60s"
straylight: lease_idle = "30m"
base_url: http://straylight.<tailnet>:11434 queue_max = 8
weight: 1.0 retention = "180d"
models: { ornith-1.5-35b-a3b: {parallel: 4}, ornith-1.5-9b-uncensored: {parallel: 6} }
titan: [hosts.straylight]
base_url: http://titan.<tailnet>:8081 base_url = "http://straylight.<tailnet>:11434"
weight: 2.0 # ~2× straylight decode weight = 1.0
models: { ornith-1.5-35b-a3b: {parallel: 4}, laguna-s-2.1: {parallel: 2} } models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "ornith-1.5-9b-uncensored" = { parallel = 6 } }
dixie:
base_url: http://dixie.<tailnet>:11434 [hosts.titan]
weight: 0.8 base_url = "http://titan.<tailnet>:8081"
models: { ornith-1.5-9b-uncensored: {parallel: 6} } weight = 2.0 # ~2x straylight decode
routes: # optional defaults / pins models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "laguna-s-2.1" = { parallel = 2 } }
opencode-a: { default_model: ornith-1.5-35b-a3b }
opencode-b: { default_model: ornith-1.5-35b-a3b } [hosts.dixie]
paper: { default_model: qwen3.8-27b-uncensored, pin: titan } base_url = "http://dixie.<tailnet>:11434"
hermes-straylight: {} weight = 0.8
hermes-talos: {} models = { "ornith-1.5-9b-uncensored" = { parallel = 6 } }
hermes-titan: { pin: titan } # titan's models are the titan agent's first
# optional per-route defaults / pins
[routes.opencode-a]
default_model = "ornith-1.5-35b-a3b"
[routes.opencode-b]
default_model = "ornith-1.5-35b-a3b"
[routes.paper]
default_model = "qwen3.8-27b-uncensored"
pin = "titan"
[routes.hermes-straylight]
[routes.hermes-talos]
[routes.hermes-titan]
pin = "titan" # titan's models are the titan agent's first
``` ```
Client side, no code changes: Client side, no code changes:
@@ -201,7 +213,7 @@ Client side, no code changes:
## 10. Milestones ## 10. Milestones
- **v0 (a day):** config, static routes → host mapping, reverse proxy with - **v0 (a day):** TOML config, static routes → host mapping, reverse proxy with
streaming, `/health` + `/models` poller, passive health, `/_crossbar/hosts`. streaming, `/health` + `/models` poller, passive health, `/_crossbar/hosts`.
Replaces the hand-maintained provider lists. No leases yet: each route has Replaces the hand-maintained provider lists. No leases yet: each route has
a fixed host list in preference order; first healthy wins. a fixed host list in preference order; first healthy wins.