deploy/hyperborea: production config, user unit, install script, README (wake block pending titan's MAC)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Claude Fable 5.1
@@ -0,0 +1,82 @@
|
||||
# crossbar on hyperborea
|
||||
|
||||
crossbar runs on **hyperborea** (Raspberry Pi, Debian 13, aarch64) as a `systemd --user` unit,
|
||||
bound to its tailnet address only. Clients on the tailnet reach it at
|
||||
|
||||
http://hyperborea.scylla-hammerhead.ts.net:7777/<route>/v1
|
||||
|
||||
Why hyperborea: it is always on, wired on titan's LAN segment (`192.168.88.154`, which
|
||||
wake-on-LAN needs — magic packets are L2 broadcast), and not itself an inference host, so a
|
||||
router rebuild or a sleeping titan never takes crossbar down with it.
|
||||
|
||||
## Files
|
||||
|
||||
| file | purpose |
|
||||
|---|---|
|
||||
| `crossbar.toml` | the production config: hosts titan/straylight/dixie with their configured models and `parallel`, the routes |
|
||||
| `crossbar.service` | the user unit (`/srv/crossbar`, `Restart=always`) |
|
||||
| `install.sh` | cross-compiles for arm64 on the machine you run it from, copies binary + config + unit, restarts, prints the hosts view |
|
||||
|
||||
On hyperborea: binary, config and SQLite database live in `/srv/crossbar/`; the unit is
|
||||
`~/.config/systemd/user/crossbar.service` (`loginctl` linger is on, so it survives logout).
|
||||
|
||||
## Install / upgrade
|
||||
|
||||
deploy/hyperborea/install.sh # from any checkout on a host with Go 1.26 and ssh to hyperborea
|
||||
|
||||
Re-running upgrades in place (binary is replaced atomically, the unit restarted; leases persist in
|
||||
the database). Config-only changes: edit `crossbar.toml`, re-run.
|
||||
|
||||
## Verify
|
||||
|
||||
curl -s http://hyperborea.scylla-hammerhead.ts.net:7777/_crossbar/hosts | jq .
|
||||
curl -s http://hyperborea.scylla-hammerhead.ts.net:7777/_crossbar/routes | jq .
|
||||
curl -s 'http://hyperborea.scylla-hammerhead.ts.net:7777/_crossbar/usage?by=route'
|
||||
ssh hyperborea journalctl --user -u crossbar -f
|
||||
|
||||
A cheap end-to-end check uses the `probe` route (dixie's 9B first):
|
||||
|
||||
curl -s -D - -X POST -H 'Content-Type: application/json' \
|
||||
-d '{"model":"ornith-1.5-9b-uncensored","max_tokens":8,"messages":[{"role":"user","content":"Reply with pong."}]}' \
|
||||
http://hyperborea.scylla-hammerhead.ts.net:7777/probe/v1/chat/completions
|
||||
|
||||
The response carries `X-Crossbar-Host` (which router served it) and `X-Crossbar-Lease`
|
||||
(`new` or `reused`).
|
||||
|
||||
## Pointing clients at it
|
||||
|
||||
OpenCode (project-local `opencode.json`, or the global one with a per-project route):
|
||||
|
||||
```jsonc
|
||||
"provider": { "crossbar": { "npm": "@ai-sdk/openai-compatible",
|
||||
"options": { "baseURL": "http://hyperborea.scylla-hammerhead.ts.net:7777/opencode-a/v1" },
|
||||
"models": { "ornith-1.5-35b-a3b": {} } } }
|
||||
```
|
||||
|
||||
Hermes (`custom_providers[].base_url`, and the same in `delegation`/`auxiliary` blocks):
|
||||
|
||||
base_url: http://hyperborea.scylla-hammerhead.ts.net:7777/hermes-straylight/v1
|
||||
|
||||
Routes must exist in `crossbar.toml`; an unknown first path segment is `404 unknown route`.
|
||||
**Known gap:** `PLAN.md`'s one-route-per-instance launcher (`CROSSBAR_ROUTE="$(basename "$PWD")-$$"`)
|
||||
needs a route *template* (e.g. `[routes."opencode-*"]`) that the code does not have yet; until
|
||||
then add each instance's route explicitly.
|
||||
|
||||
## Wake-on-LAN for titan
|
||||
|
||||
The `[hosts.titan.wake]` block is present but commented out until the MAC is settled. Titan is on
|
||||
Wi-Fi (active private address `5e:fc:f2:3f:23:6b`, hardware `60:3e:5f:33:6f:b8`) with its dock's
|
||||
three Ethernet ports (`d2:30:99:9a:ee:03/04/05`) unplugged. Wired + `womp 1` is the reliable path;
|
||||
magic-packet wake over Wi-Fi on Apple Silicon is not guaranteed and the private address may
|
||||
rotate. Broadcast address is `192.168.88.255:9`.
|
||||
|
||||
## Security notes
|
||||
|
||||
- The bind is the tailnet address; only tailnet members can reach it. `identity = "tailscale"`
|
||||
with per-route `peers` is available when a route should be limited to named nodes;
|
||||
`tailscale whois` already works unprivileged on hyperborea.
|
||||
- Plain HTTP over the tailnet is WireGuard-encrypted on the wire. Hermes agents' *terminal*
|
||||
calls to this URL may trip tirith's `plain_http_to_sink`; prefer the MagicDNS name (never the
|
||||
raw IP) and add a rule-scoped trust entry rather than `--broad` if a prompt recurs. Provider
|
||||
traffic from the OpenAI client library is not scanned by tirith.
|
||||
- Bodies are never logged or stored; the database holds leases and per-request accounting only.
|
||||
@@ -0,0 +1,18 @@
|
||||
[Unit]
|
||||
Description=crossbar — affinity router for the fleet's llama-servers (tailnet :7777)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
RequiresMountsFor=/srv
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/srv/crossbar
|
||||
ExecStart=/srv/crossbar/crossbar -config /srv/crossbar/crossbar.toml
|
||||
# The bind is the tailnet address; if tailscaled is not up yet at login, retry until it is.
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
@@ -0,0 +1,65 @@
|
||||
# crossbar on hyperborea — the fleet's llama-server routers behind one tailnet endpoint.
|
||||
# Clients: http://hyperborea.scylla-hammerhead.ts.net:7777/<route>/v1
|
||||
listen = "100.112.40.10:7777" # hyperborea's tailnet address only; never a LAN or 0.0.0.0 bind
|
||||
db = "/srv/crossbar/crossbar.db"
|
||||
poll_interval = "60s"
|
||||
lease_idle = "30m"
|
||||
retention = "180d"
|
||||
queue_max = 2 # waiting places per (host, model) beyond `parallel`; 503 past that
|
||||
identity = "off" # switch to "tailscale" once routes carry `peers`
|
||||
|
||||
# `models` lists what each router is configured to serve, with that model's `parallel` from its
|
||||
# preset; the poller learns which are actually loaded (only those count for stickiness and the
|
||||
# context guard) and a request for an unloaded model still goes to a healthy host, where the
|
||||
# router autoloads it as today.
|
||||
|
||||
[hosts.titan] # M3 Max 128 GB; ~2x straylight's decode speed
|
||||
base_url = "http://titan.scylla-hammerhead.ts.net:8081"
|
||||
weight = 2.0
|
||||
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "ornith-1.5-9b-uncensored" = { parallel = 2 }, "qwen3.8-27b-uncensored" = { parallel = 2 }, "qwen3.6-35b-a3b-abliterated" = { parallel = 2 }, "gemma4-26b-a4b-abliterated" = { parallel = 2 }, "laguna-s-2.1" = { parallel = 2 }, "hermes4-70b-heretic" = { parallel = 1 }, "llama33-70b-abliterated" = { parallel = 1 }, "qwen25-72b-abliterated" = { parallel = 1 } }
|
||||
# Wake-on-LAN: enable once titan's wake MAC is decided (see README). Magic packets are L2
|
||||
# broadcast; hyperborea is wired on titan's segment (192.168.88.0/24).
|
||||
# [hosts.titan.wake]
|
||||
# mac = "d2:30:99:9a:ee:03" # dock Ethernet en4 if titan is wired; "5e:fc:f2:3f:23:6b" is the Wi-Fi private address
|
||||
# broadcast = "192.168.88.255:9"
|
||||
# wait = "45s"
|
||||
|
||||
[hosts.straylight]
|
||||
base_url = "http://straylight.scylla-hammerhead.ts.net:11434"
|
||||
weight = 1.0
|
||||
models = { "ornith-1.5-35b-a3b" = { parallel = 4 }, "ornith-1.5-9b-uncensored" = { parallel = 2 }, "qwen3.8-27b-uncensored" = { parallel = 2 }, "qwen3.6-35b-a3b-abliterated" = { parallel = 2 }, "gemma4-26b-a4b-abliterated" = { parallel = 2 }, "qwen3-vl-8b-abliterated" = { parallel = 2 }, "qwen3.8-flash-next-uncensored" = { parallel = 1 }, "ornith-1.0-35b" = { parallel = 2 } }
|
||||
|
||||
[hosts.dixie] # helper tier: the 9B only (honcho-embed is Honcho's lane, not routed)
|
||||
base_url = "http://dixie.scylla-hammerhead.ts.net:11434"
|
||||
weight = 0.5
|
||||
models = { "ornith-1.5-9b-uncensored" = { parallel = 8 } }
|
||||
|
||||
# Routes: the first URL path segment (or X-Crossbar-Route). Each conversation on a route gets a
|
||||
# sticky lease on the host with the most free slots x weight when it starts.
|
||||
[routes.opencode-a]
|
||||
hosts = ["titan", "straylight"]
|
||||
default_model = "ornith-1.5-35b-a3b"
|
||||
|
||||
[routes.opencode-b]
|
||||
hosts = ["titan", "straylight"]
|
||||
default_model = "ornith-1.5-35b-a3b"
|
||||
|
||||
[routes.paper]
|
||||
hosts = ["titan", "straylight"]
|
||||
default_model = "ornith-1.5-35b-a3b"
|
||||
|
||||
[routes.hermes-straylight]
|
||||
hosts = ["straylight", "titan", "dixie"]
|
||||
default_model = "ornith-1.5-35b-a3b"
|
||||
|
||||
[routes.hermes-titan]
|
||||
hosts = ["titan", "straylight", "dixie"]
|
||||
default_model = "ornith-1.5-35b-a3b"
|
||||
|
||||
[routes.hermes-talos]
|
||||
hosts = ["titan", "straylight", "dixie"]
|
||||
default_model = "ornith-1.5-35b-a3b"
|
||||
|
||||
[routes.probe] # for operators: curl tests, never a real client
|
||||
hosts = ["dixie", "straylight", "titan"]
|
||||
default_model = "ornith-1.5-9b-uncensored"
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/bin/sh
|
||||
# Build crossbar for hyperborea (arm64, static) on this machine and install it there as a
|
||||
# systemd --user unit. Run from anywhere inside the repo. Idempotent: re-running upgrades in place.
|
||||
set -eu
|
||||
HOST=${HOST:-hyperborea}
|
||||
DIR=/srv/crossbar
|
||||
cd "$(git rev-parse --show-toplevel)"
|
||||
out=$(mktemp -t crossbar-arm64.XXXXXX)
|
||||
trap 'rm -f "$out"' EXIT
|
||||
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -trimpath -ldflags="-s -w" -o "$out" ./cmd/crossbar
|
||||
ssh "$HOST" "mkdir -p $DIR ~/.config/systemd/user"
|
||||
scp -q "$out" "$HOST:$DIR/crossbar.new"
|
||||
scp -q deploy/hyperborea/crossbar.toml "$HOST:$DIR/crossbar.toml"
|
||||
scp -q deploy/hyperborea/crossbar.service "$HOST:.config/systemd/user/crossbar.service"
|
||||
ssh "$HOST" "chmod 755 $DIR/crossbar.new && mv $DIR/crossbar.new $DIR/crossbar \
|
||||
&& systemctl --user daemon-reload && systemctl --user enable crossbar.service >/dev/null 2>&1 \
|
||||
&& systemctl --user restart crossbar.service && sleep 2 && systemctl --user is-active crossbar.service"
|
||||
echo "installed; hosts view:"
|
||||
curl -fsS "http://$HOST.scylla-hammerhead.ts.net:7777/_crossbar/hosts"
|
||||
echo
|
||||
Reference in New Issue
Block a user