crossbar/README.md

crossbar

crossbar is an affinity router in front of several llama-server routers. A client's identity is the first path segment of its base URL; v0 routes each request to the first healthy host on that route's list and streams the answer back unbuffered.

Build

Build everything with make build; the binaries land in bin/. Check the work with make gate, which runs the formatter, vet, tests and line-length check with no network. When the code is ready, run make smoke, which starts two fake upstreams and exercises routing, failover, recovery and streaming over real HTTP.

Configure

crossbar reads one TOML file. This is example.toml:

# crossbar example configuration. Replace <tailnet> and the addresses with your own.
listen        = "127.0.0.1:17777"   # never 0.0.0.0 — bind the tailnet address in production
poll_interval = "1s"                # 60s in production; 1s makes the smoke run quick
queue_max     = 8

[hosts.alpha]
base_url = "http://127.0.0.1:18081"    # e.g. http://straylight.<tailnet>:11434
weight   = 1.0
models   = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "small-9b" = { parallel = 6 } }

[hosts.beta]
base_url = "http://127.0.0.1:18082"    # e.g. http://titan.<tailnet>:8081
weight   = 2.0
models   = { "ornith-1.5-35b-a3b" = { parallel = 4 } }

# v0: a route is a preference list; the first healthy host that has the model wins.
[routes.opencode-a]
hosts         = ["alpha", "beta"]
default_model = "ornith-1.5-35b-a3b"

[routes.hermes-x]
hosts = ["beta", "alpha"]

Key	Meaning
listen	Where crossbar binds. A tailnet address, never 0.0.0.0.
poll_interval	How often each host is health-checked. 60s in production; 1s makes the smoke run quick.
queue_max	Reserved for v1 queueing; no effect in v0.
hosts.<name>.base_url	The llama-server base URL this host serves.
hosts.<name>.weight	Relative share of new routes this host receives.
hosts.<name>.models	The models this host serves, with per-model parallel tuning.
routes.<name>.hosts	Preference order: the first healthy host that serves the model wins.
routes.<name>.default_model	Model used when a request omits one; must be served by a host in the route.

Run

Copy the binary, the config and the unit into place, reload systemd, and start it:

install -m 0755 bin/crossbar /usr/local/bin/crossbar
install -d -m 0755 /etc/crossbar
install -m 0644 crossbar.toml /etc/crossbar/crossbar.toml
install -m 0644 deploy/crossbar.service /etc/systemd/system/crossbar.service
systemctl daemon-reload
systemctl enable --now crossbar

Point clients at it

OpenCode, one provider for every project. Each instance is launched as CROSSBAR_ROUTE="$(basename "$PWD")-$$" opencode:

"provider": { "crossbar": { "npm": "@ai-sdk/openai-compatible",
  "options": { "baseURL": "http://crossbar.<tailnet>:7777/{env:CROSSBAR_ROUTE}/v1" },
  "models": { "ornith-1.5-35b-a3b": {} } } }

Hermes, in config.yaml:

custom_providers:
  - name: crossbar
    base_url: http://crossbar.<tailnet>:7777/hermes-<agent>/v1
    models: { ornith-1.5-35b-a3b: {} }

The route name in the URL must exist in [routes]; unknown routes are 404.

Inspect

GET /_crossbar/hosts reports every host's health and loaded models:

{"alpha":{"healthy":true,"loaded":["ornith-1.5-35b-a3b","small-9b"],"last_ok":"2026-09-25T09:34:18Z","last_err":""},"beta":{"healthy":true,"loaded":["ornith-1.5-35b-a3b"],"last_ok":"2026-09-25T09:34:18Z","last_err":""}}

GET /_crossbar/routes reports each route's preference order and default model:

{"hermes-x":{"hosts":["beta","alpha"],"default_model":""},"opencode-a":{"hosts":["alpha","beta"],"default_model":"ornith-1.5-35b-a3b"}}

What v0 does not do

Leases and stickiness, SQLite, /slots, queueing and wake-on-LAN are out of scope for v0; see PLAN.md.