Add the smoke run, README and systemd unit
Implemented-By: OpenCode session (model recorded in docs/implementer-log.md)
This commit is contained in:
@@ -0,0 +1,106 @@
|
|||||||
|
# crossbar
|
||||||
|
|
||||||
|
crossbar is an affinity router in front of several `llama-server` routers. A client's identity is
|
||||||
|
the first path segment of its base URL; v0 routes each request to the first healthy host on that
|
||||||
|
route's list and streams the answer back unbuffered.
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
Build everything with `make build`; the binaries land in `bin/`. Check the work with `make gate`,
|
||||||
|
which runs the formatter, vet, tests and line-length check with no network. When the code is ready,
|
||||||
|
run `make smoke`, which starts two fake upstreams and exercises routing, failover, recovery and
|
||||||
|
streaming over real HTTP.
|
||||||
|
|
||||||
|
## Configure
|
||||||
|
|
||||||
|
crossbar reads one TOML file. This is `example.toml`:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
# crossbar example configuration. Replace <tailnet> and the addresses with your own.
|
||||||
|
listen = "127.0.0.1:17777" # never 0.0.0.0 — bind the tailnet address in production
|
||||||
|
poll_interval = "1s" # 60s in production; 1s makes the smoke run quick
|
||||||
|
queue_max = 8
|
||||||
|
|
||||||
|
[hosts.alpha]
|
||||||
|
base_url = "http://127.0.0.1:18081" # e.g. http://straylight.<tailnet>:11434
|
||||||
|
weight = 1.0
|
||||||
|
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "small-9b" = { parallel = 6 } }
|
||||||
|
|
||||||
|
[hosts.beta]
|
||||||
|
base_url = "http://127.0.0.1:18082" # e.g. http://titan.<tailnet>:8081
|
||||||
|
weight = 2.0
|
||||||
|
models = { "ornith-1.5-35b-a3b" = { parallel = 4 } }
|
||||||
|
|
||||||
|
# v0: a route is a preference list; the first healthy host that has the model wins.
|
||||||
|
[routes.opencode-a]
|
||||||
|
hosts = ["alpha", "beta"]
|
||||||
|
default_model = "ornith-1.5-35b-a3b"
|
||||||
|
|
||||||
|
[routes.hermes-x]
|
||||||
|
hosts = ["beta", "alpha"]
|
||||||
|
```
|
||||||
|
|
||||||
|
| Key | Meaning |
|
||||||
|
| --- | --- |
|
||||||
|
| `listen` | Where crossbar binds. A tailnet address, never `0.0.0.0`. |
|
||||||
|
| `poll_interval` | How often each host is health-checked. 60s in production; 1s makes the smoke run quick. |
|
||||||
|
| `queue_max` | Reserved for v1 queueing; no effect in v0. |
|
||||||
|
| `hosts.<name>.base_url` | The llama-server base URL this host serves. |
|
||||||
|
| `hosts.<name>.weight` | Relative share of new routes this host receives. |
|
||||||
|
| `hosts.<name>.models` | The models this host serves, with per-model parallel tuning. |
|
||||||
|
| `routes.<name>.hosts` | Preference order: the first healthy host that serves the model wins. |
|
||||||
|
| `routes.<name>.default_model` | Model used when a request omits one; must be served by a host in the route. |
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
Copy the binary, the config and the unit into place, reload systemd, and start it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
install -m 0755 bin/crossbar /usr/local/bin/crossbar
|
||||||
|
install -d -m 0755 /etc/crossbar
|
||||||
|
install -m 0644 crossbar.toml /etc/crossbar/crossbar.toml
|
||||||
|
install -m 0644 deploy/crossbar.service /etc/systemd/system/crossbar.service
|
||||||
|
systemctl daemon-reload
|
||||||
|
systemctl enable --now crossbar
|
||||||
|
```
|
||||||
|
|
||||||
|
## Point clients at it
|
||||||
|
|
||||||
|
OpenCode, one provider for every project. Each instance is launched as
|
||||||
|
`CROSSBAR_ROUTE="$(basename "$PWD")-$$" opencode`:
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
"provider": { "crossbar": { "npm": "@ai-sdk/openai-compatible",
|
||||||
|
"options": { "baseURL": "http://crossbar.<tailnet>:7777/{env:CROSSBAR_ROUTE}/v1" },
|
||||||
|
"models": { "ornith-1.5-35b-a3b": {} } } }
|
||||||
|
```
|
||||||
|
|
||||||
|
Hermes, in `config.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
custom_providers:
|
||||||
|
- name: crossbar
|
||||||
|
base_url: http://crossbar.<tailnet>:7777/hermes-<agent>/v1
|
||||||
|
models: { ornith-1.5-35b-a3b: {} }
|
||||||
|
```
|
||||||
|
|
||||||
|
The route name in the URL must exist in `[routes]`; unknown routes are 404.
|
||||||
|
|
||||||
|
## Inspect
|
||||||
|
|
||||||
|
`GET /_crossbar/hosts` reports every host's health and loaded models:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"alpha":{"healthy":true,"loaded":["ornith-1.5-35b-a3b","small-9b"],"last_ok":"2026-09-25T09:34:18Z","last_err":""},"beta":{"healthy":true,"loaded":["ornith-1.5-35b-a3b"],"last_ok":"2026-09-25T09:34:18Z","last_err":""}}
|
||||||
|
```
|
||||||
|
|
||||||
|
`GET /_crossbar/routes` reports each route's preference order and default model:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"hermes-x":{"hosts":["beta","alpha"],"default_model":""},"opencode-a":{"hosts":["alpha","beta"],"default_model":"ornith-1.5-35b-a3b"}}
|
||||||
|
```
|
||||||
|
|
||||||
|
## What v0 does not do
|
||||||
|
|
||||||
|
Leases and stickiness, SQLite, `/slots`, queueing and wake-on-LAN are out of scope for v0; see
|
||||||
|
`PLAN.md`.
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=crossbar affinity router for llama-server
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
ExecStart=/usr/local/bin/crossbar -config /etc/crossbar/crossbar.toml
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=2s
|
||||||
|
DynamicUser=yes
|
||||||
|
StateDirectory=crossbar
|
||||||
|
NoNewPrivileges=yes
|
||||||
|
ProtectSystem=strict
|
||||||
|
ProtectHome=yes
|
||||||
|
PrivateTmp=yes
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
@@ -9,5 +9,6 @@ owner fills in the Model column. The reviewer adds findings under "Reviews" once
|
|||||||
| v0/02-health | 2026-09-25 | done | 1 | pass | none | First gate run passed. `MarkDown` initially forgot to write the entry back; caught by `TestMarkDown`. | ? |
|
| v0/02-health | 2026-09-25 | done | 1 | pass | none | First gate run passed. `MarkDown` initially forgot to write the entry back; caught by `TestMarkDown`. | ? |
|
||||||
| v0/03-proxy | 2026-09-25 | done | 1 | pass | none | `SplitRoute` must reject an empty first segment (`/`, `//x`) as `ok=false`; the model peek restores the body and leaves non-JSON/empty as `""`. | ? |
|
| v0/03-proxy | 2026-09-25 | done | 1 | pass | none | `SplitRoute` must reject an empty first segment (`/`, `//x`) as `ok=false`; the model peek restores the body and leaves non-JSON/empty as `""`. | ? |
|
||||||
| v0/04-admin-main | 2026-09-25 | done | 1 | pass | none | `timeout --signal=TERM 3` exits 124 on a timed-out child on this GNU system, so the task's `exit=0` is not observable through it; sent SIGTERM directly and confirmed crossbar's own exit code is 0 with both log lines. | ? |
|
| v0/04-admin-main | 2026-09-25 | done | 1 | pass | none | `timeout --signal=TERM 3` exits 124 on a timed-out child on this GNU system, so the task's `exit=0` is not observable through it; sent SIGTERM directly and confirmed crossbar's own exit code is 0 with both log lines. | ? |
|
||||||
|
| v0/05-smoke-readme-deploy | 2026-09-25 | done | 1 | pass | none | `README.md` `## Run` uses `install -m` instead of `cp` and adds `systemctl daemon-reload` before `enable --now`, which is required for systemd to see the new unit; the task said only "copy … then enable --now". | ? |
|
||||||
|
|
||||||
## Reviews
|
## Reviews
|
||||||
|
|||||||
Executable
+45
@@ -0,0 +1,45 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# Smoke run: two fake upstreams, one crossbar, real HTTP. Prints "smoke: ok" or fails.
|
||||||
|
# Needs: bin/crossbar and bin/fakeupstream (make build), curl.
|
||||||
|
set -eu
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
tmp=$(mktemp -d); trap 'kill $pids 2>/dev/null; rm -rf "$tmp"' EXIT INT TERM
|
||||||
|
pids=""
|
||||||
|
bin/fakeupstream -listen 127.0.0.1:18081 -name alpha -models ornith-1.5-35b-a3b,small-9b -down-file "$tmp/alpha.down" >"$tmp/alpha.log" 2>&1 & pids="$pids $!"
|
||||||
|
bin/fakeupstream -listen 127.0.0.1:18082 -name beta -models ornith-1.5-35b-a3b -down-file "$tmp/beta.down" >"$tmp/beta.log" 2>&1 & pids="$pids $!"
|
||||||
|
bin/crossbar -config example.toml >"$tmp/crossbar.log" 2>&1 & pids="$pids $!"
|
||||||
|
sleep 1.5
|
||||||
|
fail() { echo "smoke: FAIL: $*" >&2; echo "--- crossbar.log"; cat "$tmp/crossbar.log"; exit 1; }
|
||||||
|
base=http://127.0.0.1:17777
|
||||||
|
|
||||||
|
h=$(curl -s -o /dev/null -w '%{http_code} %header{X-Crossbar-Host}' "$base/opencode-a/v1/models")
|
||||||
|
[ "$h" = "200 alpha" ] || fail "opencode-a should go to alpha, got '$h'"
|
||||||
|
h=$(curl -s -o /dev/null -w '%{http_code} %header{X-Crossbar-Host}' "$base/hermes-x/v1/models")
|
||||||
|
[ "$h" = "200 beta" ] || fail "hermes-x should go to beta, got '$h'"
|
||||||
|
h=$(curl -s -o /dev/null -w '%{http_code}' "$base/nope/v1/models")
|
||||||
|
[ "$h" = "404" ] || fail "unknown route should be 404, got '$h'"
|
||||||
|
|
||||||
|
touch "$tmp/alpha.down"; sleep 2.5 # poll_interval is 1s in example.toml
|
||||||
|
h=$(curl -s -o /dev/null -w '%{http_code} %header{X-Crossbar-Host}' "$base/opencode-a/v1/models")
|
||||||
|
[ "$h" = "200 beta" ] || fail "with alpha down, opencode-a should fail over to beta, got '$h'"
|
||||||
|
curl -s "$base/_crossbar/hosts" | grep -q '"alpha":{"healthy":false' || fail "/_crossbar/hosts does not show alpha unhealthy: $(curl -s $base/_crossbar/hosts)"
|
||||||
|
|
||||||
|
rm "$tmp/alpha.down"; sleep 3.5 # recovery needs two good polls
|
||||||
|
h=$(curl -s -o /dev/null -w '%header{X-Crossbar-Host}' "$base/opencode-a/v1/models")
|
||||||
|
[ "$h" = "alpha" ] || fail "alpha should be back after two good polls, got '$h'"
|
||||||
|
|
||||||
|
# Streaming: five chunks 200 ms apart must arrive over >= 0.6 s, not all at once at the end.
|
||||||
|
start=$(date +%s%N)
|
||||||
|
first=""
|
||||||
|
curl -sN -X POST -H 'Content-Type: application/json' -d '{"model":"ornith-1.5-35b-a3b","stream":true,"messages":[]}' \
|
||||||
|
"$base/opencode-a/v1/chat/completions" | while IFS= read -r line; do
|
||||||
|
[ -n "$line" ] || continue
|
||||||
|
now=$(date +%s%N); echo "$(( (now - start) / 1000000 )) $line"
|
||||||
|
done > "$tmp/stream.txt"
|
||||||
|
firstms=$(head -1 "$tmp/stream.txt" | cut -d' ' -f1); lastms=$(tail -1 "$tmp/stream.txt" | cut -d' ' -f1)
|
||||||
|
[ -n "$firstms" ] && [ "$((lastms - firstms))" -ge 600 ] || fail "stream arrived in one burst (first ${firstms:-?} ms, last ${lastms:-?} ms):
|
||||||
|
$(cat "$tmp/stream.txt")"
|
||||||
|
grep -q 'DONE' "$tmp/stream.txt" || fail "stream did not end with [DONE]"
|
||||||
|
|
||||||
|
grep -q 'route=opencode-a host=alpha' "$tmp/crossbar.log" || fail "no request log line"
|
||||||
|
echo "smoke: ok (stream spread $((lastms - firstms)) ms)"
|
||||||
Reference in New Issue
Block a user