PLAN rev 3: TOML config
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# crossbar — an affinity router for the fleet's llama-servers
|
||||
|
||||
**Status:** plan, 2026-09-25 (rev 2: SQLite state + accounting, Kyle). Nothing built. Go.
|
||||
**Status:** plan, 2026-09-25 (rev 2: SQLite state + accounting; rev 3: TOML config — Kyle). Nothing built. Go.
|
||||
**Owner:** Kyle. Drafted by claude from the 2026-09-25 discussion (Inference Infrastructure).
|
||||
|
||||
## 1. Problem
|
||||
@@ -145,35 +145,47 @@ exposes the rollups; a nightly job prunes `requests` older than `retention`
|
||||
|
||||
Body contents are never stored — only counts and timings.
|
||||
|
||||
## 8. Config (`crossbar.yaml`)
|
||||
## 8. Config (`crossbar.toml`)
|
||||
|
||||
```yaml
|
||||
listen: "100.x.y.z:7777" # tailnet address only; never 0.0.0.0
|
||||
db: /var/lib/crossbar/crossbar.db
|
||||
retention: 180d
|
||||
poll_interval: 60s
|
||||
lease_idle: 30m
|
||||
queue_max: 8
|
||||
hosts:
|
||||
straylight:
|
||||
base_url: http://straylight.<tailnet>:11434
|
||||
weight: 1.0
|
||||
models: { ornith-1.5-35b-a3b: {parallel: 4}, ornith-1.5-9b-uncensored: {parallel: 6} }
|
||||
titan:
|
||||
base_url: http://titan.<tailnet>:8081
|
||||
weight: 2.0 # ~2× straylight decode
|
||||
models: { ornith-1.5-35b-a3b: {parallel: 4}, laguna-s-2.1: {parallel: 2} }
|
||||
dixie:
|
||||
base_url: http://dixie.<tailnet>:11434
|
||||
weight: 0.8
|
||||
models: { ornith-1.5-9b-uncensored: {parallel: 6} }
|
||||
routes: # optional defaults / pins
|
||||
opencode-a: { default_model: ornith-1.5-35b-a3b }
|
||||
opencode-b: { default_model: ornith-1.5-35b-a3b }
|
||||
paper: { default_model: qwen3.8-27b-uncensored, pin: titan }
|
||||
hermes-straylight: {}
|
||||
hermes-talos: {}
|
||||
hermes-titan: { pin: titan } # titan's models are the titan agent's first
|
||||
TOML (`github.com/BurntSushi/toml`): no indentation semantics, no implicit
|
||||
type coercion, and it is what Kyle's Rust projects already use. A NixOS module
|
||||
can generate it with `pkgs.formats.toml`.
|
||||
|
||||
```toml
|
||||
listen = "100.x.y.z:7777" # tailnet address only; never 0.0.0.0
|
||||
db = "/var/lib/crossbar/crossbar.db"
|
||||
poll_interval = "60s"
|
||||
lease_idle = "30m"
|
||||
queue_max = 8
|
||||
retention = "180d"
|
||||
|
||||
[hosts.straylight]
|
||||
base_url = "http://straylight.<tailnet>:11434"
|
||||
weight = 1.0
|
||||
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "ornith-1.5-9b-uncensored" = { parallel = 6 } }
|
||||
|
||||
[hosts.titan]
|
||||
base_url = "http://titan.<tailnet>:8081"
|
||||
weight = 2.0 # ~2x straylight decode
|
||||
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "laguna-s-2.1" = { parallel = 2 } }
|
||||
|
||||
[hosts.dixie]
|
||||
base_url = "http://dixie.<tailnet>:11434"
|
||||
weight = 0.8
|
||||
models = { "ornith-1.5-9b-uncensored" = { parallel = 6 } }
|
||||
|
||||
# optional per-route defaults / pins
|
||||
[routes.opencode-a]
|
||||
default_model = "ornith-1.5-35b-a3b"
|
||||
[routes.opencode-b]
|
||||
default_model = "ornith-1.5-35b-a3b"
|
||||
[routes.paper]
|
||||
default_model = "qwen3.8-27b-uncensored"
|
||||
pin = "titan"
|
||||
[routes.hermes-straylight]
|
||||
[routes.hermes-talos]
|
||||
[routes.hermes-titan]
|
||||
pin = "titan" # titan's models are the titan agent's first
|
||||
```
|
||||
|
||||
Client side, no code changes:
|
||||
@@ -201,7 +213,7 @@ Client side, no code changes:
|
||||
|
||||
## 10. Milestones
|
||||
|
||||
- **v0 (a day):** config, static routes → host mapping, reverse proxy with
|
||||
- **v0 (a day):** TOML config, static routes → host mapping, reverse proxy with
|
||||
streaming, `/health` + `/models` poller, passive health, `/_crossbar/hosts`.
|
||||
Replaces the hand-maintained provider lists. No leases yet: each route has
|
||||
a fixed host list in preference order; first healthy wins.
|
||||
|
||||
Reference in New Issue
Block a user