PLAN rev 3: TOML config
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,6 @@
|
|||||||
# crossbar — an affinity router for the fleet's llama-servers
|
# crossbar — an affinity router for the fleet's llama-servers
|
||||||
|
|
||||||
**Status:** plan, 2026-09-25 (rev 2: SQLite state + accounting, Kyle). Nothing built. Go.
|
**Status:** plan, 2026-09-25 (rev 2: SQLite state + accounting; rev 3: TOML config — Kyle). Nothing built. Go.
|
||||||
**Owner:** Kyle. Drafted by claude from the 2026-09-25 discussion (Inference Infrastructure).
|
**Owner:** Kyle. Drafted by claude from the 2026-09-25 discussion (Inference Infrastructure).
|
||||||
|
|
||||||
## 1. Problem
|
## 1. Problem
|
||||||
@@ -145,35 +145,47 @@ exposes the rollups; a nightly job prunes `requests` older than `retention`
|
|||||||
|
|
||||||
Body contents are never stored — only counts and timings.
|
Body contents are never stored — only counts and timings.
|
||||||
|
|
||||||
## 8. Config (`crossbar.yaml`)
|
## 8. Config (`crossbar.toml`)
|
||||||
|
|
||||||
```yaml
|
TOML (`github.com/BurntSushi/toml`): no indentation semantics, no implicit
|
||||||
listen: "100.x.y.z:7777" # tailnet address only; never 0.0.0.0
|
type coercion, and it is what Kyle's Rust projects already use. A NixOS module
|
||||||
db: /var/lib/crossbar/crossbar.db
|
can generate it with `pkgs.formats.toml`.
|
||||||
retention: 180d
|
|
||||||
poll_interval: 60s
|
```toml
|
||||||
lease_idle: 30m
|
listen = "100.x.y.z:7777" # tailnet address only; never 0.0.0.0
|
||||||
queue_max: 8
|
db = "/var/lib/crossbar/crossbar.db"
|
||||||
hosts:
|
poll_interval = "60s"
|
||||||
straylight:
|
lease_idle = "30m"
|
||||||
base_url: http://straylight.<tailnet>:11434
|
queue_max = 8
|
||||||
weight: 1.0
|
retention = "180d"
|
||||||
models: { ornith-1.5-35b-a3b: {parallel: 4}, ornith-1.5-9b-uncensored: {parallel: 6} }
|
|
||||||
titan:
|
[hosts.straylight]
|
||||||
base_url: http://titan.<tailnet>:8081
|
base_url = "http://straylight.<tailnet>:11434"
|
||||||
weight: 2.0 # ~2× straylight decode
|
weight = 1.0
|
||||||
models: { ornith-1.5-35b-a3b: {parallel: 4}, laguna-s-2.1: {parallel: 2} }
|
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "ornith-1.5-9b-uncensored" = { parallel = 6 } }
|
||||||
dixie:
|
|
||||||
base_url: http://dixie.<tailnet>:11434
|
[hosts.titan]
|
||||||
weight: 0.8
|
base_url = "http://titan.<tailnet>:8081"
|
||||||
models: { ornith-1.5-9b-uncensored: {parallel: 6} }
|
weight = 2.0 # ~2x straylight decode
|
||||||
routes: # optional defaults / pins
|
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "laguna-s-2.1" = { parallel = 2 } }
|
||||||
opencode-a: { default_model: ornith-1.5-35b-a3b }
|
|
||||||
opencode-b: { default_model: ornith-1.5-35b-a3b }
|
[hosts.dixie]
|
||||||
paper: { default_model: qwen3.8-27b-uncensored, pin: titan }
|
base_url = "http://dixie.<tailnet>:11434"
|
||||||
hermes-straylight: {}
|
weight = 0.8
|
||||||
hermes-talos: {}
|
models = { "ornith-1.5-9b-uncensored" = { parallel = 6 } }
|
||||||
hermes-titan: { pin: titan } # titan's models are the titan agent's first
|
|
||||||
|
# optional per-route defaults / pins
|
||||||
|
[routes.opencode-a]
|
||||||
|
default_model = "ornith-1.5-35b-a3b"
|
||||||
|
[routes.opencode-b]
|
||||||
|
default_model = "ornith-1.5-35b-a3b"
|
||||||
|
[routes.paper]
|
||||||
|
default_model = "qwen3.8-27b-uncensored"
|
||||||
|
pin = "titan"
|
||||||
|
[routes.hermes-straylight]
|
||||||
|
[routes.hermes-talos]
|
||||||
|
[routes.hermes-titan]
|
||||||
|
pin = "titan" # titan's models are the titan agent's first
|
||||||
```
|
```
|
||||||
|
|
||||||
Client side, no code changes:
|
Client side, no code changes:
|
||||||
@@ -201,7 +213,7 @@ Client side, no code changes:
|
|||||||
|
|
||||||
## 10. Milestones
|
## 10. Milestones
|
||||||
|
|
||||||
- **v0 (a day):** config, static routes → host mapping, reverse proxy with
|
- **v0 (a day):** TOML config, static routes → host mapping, reverse proxy with
|
||||||
streaming, `/health` + `/models` poller, passive health, `/_crossbar/hosts`.
|
streaming, `/health` + `/models` poller, passive health, `/_crossbar/hosts`.
|
||||||
Replaces the hand-maintained provider lists. No leases yet: each route has
|
Replaces the hand-maintained provider lists. No leases yet: each route has
|
||||||
a fixed host list in preference order; first healthy wins.
|
a fixed host list in preference order; first healthy wins.
|
||||||
|
|||||||
Reference in New Issue
Block a user