119 lines
4.6 KiB
Markdown
119 lines
4.6 KiB
Markdown
# v0 task 05: the smoke run, README and systemd unit
|
||
|
||
**Branch:** `v0` (run `git switch v0`; `git status --short` must be empty, otherwise stop)
|
||
**Commit subject:** `Add the smoke run, README and systemd unit`
|
||
|
||
## Goal
|
||
|
||
Prove the whole binary works over real HTTP against two fake upstreams — routing, failover,
|
||
recovery, streaming — with the given `tools/smoke.sh`, then document how to build, configure,
|
||
run and point clients at crossbar.
|
||
|
||
## Context
|
||
|
||
`tools/smoke.sh` starts two `fakeupstream`s on `127.0.0.1:18081/18082` and `bin/crossbar` on
|
||
`example.toml` (`poll_interval = "1s"`), then checks with `curl`: `opencode-a` goes to `alpha`
|
||
and `hermes-x` to `beta`; an unknown route is 404; after `alpha` is taken down (its `-down-file`
|
||
appears) requests fail over to `beta` and `/_crossbar/hosts` shows `alpha` unhealthy; after two
|
||
good polls `alpha` is back; a streamed completion of five chunks 200 ms apart reaches the client
|
||
spread over at least 600 ms, not in one burst; and crossbar wrote a request log line. It prints
|
||
`smoke: ok (stream spread N ms)` or fails with the crossbar log.
|
||
|
||
## Files
|
||
|
||
- Copy (never edit): `tools/smoke.sh`
|
||
- Create: `README.md`, `deploy/crossbar.service`
|
||
- Modify: `docs/implementer-log.md`
|
||
|
||
## Steps
|
||
|
||
- [ ] **1. Copy and run the smoke test.**
|
||
|
||
```sh
|
||
git switch v0
|
||
mkdir -p tools deploy
|
||
cp docs/plans/v0/_files/tools/smoke.sh tools/
|
||
make smoke
|
||
```
|
||
|
||
Expected last line: `smoke: ok (stream spread N ms)` with N ≥ 600. If it fails, the message says
|
||
which check failed and prints crossbar's log; the fault is in code from tasks 02–04 or in this
|
||
machine's `curl`. Fix code only if a rule from an earlier task was broken; otherwise stop and
|
||
report.
|
||
|
||
- [ ] **2. Write `deploy/crossbar.service`** with exactly this content:
|
||
|
||
```ini
|
||
[Unit]
|
||
Description=crossbar affinity router for llama-server
|
||
After=network-online.target
|
||
Wants=network-online.target
|
||
|
||
[Service]
|
||
ExecStart=/usr/local/bin/crossbar -config /etc/crossbar/crossbar.toml
|
||
Restart=on-failure
|
||
RestartSec=2s
|
||
DynamicUser=yes
|
||
StateDirectory=crossbar
|
||
NoNewPrivileges=yes
|
||
ProtectSystem=strict
|
||
ProtectHome=yes
|
||
PrivateTmp=yes
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
```
|
||
|
||
- [ ] **3. Write `README.md`** with these sections, in this order, in plain prose (no
|
||
hostnames, addresses or tokens other than the placeholders shown):
|
||
1. `# crossbar` — two sentences: an affinity router in front of several `llama-server`
|
||
routers; a client's identity is the first path segment of its base URL; v0 routes each
|
||
to the first healthy host on that route's list and streams answers unbuffered.
|
||
2. `## Build` — `make build` (binaries in `bin/`), `make gate`, `make smoke`.
|
||
3. `## Configure` — paste `example.toml` in a fenced block and explain each key in a table:
|
||
`listen` (a tailnet address, never `0.0.0.0`), `poll_interval` (60s in production),
|
||
`queue_max` (reserved for v1), `hosts.<name>.base_url|weight|models`, `routes.<name>.hosts`
|
||
(preference order) and `default_model`.
|
||
4. `## Run` — copy `bin/crossbar` to `/usr/local/bin/`, the config to
|
||
`/etc/crossbar/crossbar.toml`, the unit to `/etc/systemd/system/`, then
|
||
`systemctl enable --now crossbar`.
|
||
5. `## Point clients at it` — these two snippets verbatim:
|
||
|
||
OpenCode, one provider for every project; each instance is launched as
|
||
`CROSSBAR_ROUTE="$(basename "$PWD")-$$" opencode`:
|
||
```jsonc
|
||
"provider": { "crossbar": { "npm": "@ai-sdk/openai-compatible",
|
||
"options": { "baseURL": "http://crossbar.<tailnet>:7777/{env:CROSSBAR_ROUTE}/v1" },
|
||
"models": { "ornith-1.5-35b-a3b": {} } } }
|
||
```
|
||
Hermes, in `config.yaml`:
|
||
```yaml
|
||
custom_providers:
|
||
- name: crossbar
|
||
base_url: http://crossbar.<tailnet>:7777/hermes-<agent>/v1
|
||
models: { ornith-1.5-35b-a3b: {} }
|
||
```
|
||
Note under them: the route name in the URL must exist in `[routes]`; unknown routes are 404.
|
||
6. `## Inspect` — `GET /_crossbar/hosts` and `GET /_crossbar/routes`, one example response each
|
||
(take them from the smoke run).
|
||
7. `## What v0 does not do` — leases and stickiness, SQLite, `/slots`, queueing, wake-on-LAN:
|
||
see `PLAN.md`.
|
||
|
||
- [ ] **4. Run the gate.** `make gate`. Expected last line: `gate: ok`.
|
||
- [ ] **5. Log and commit.** Row `v0/05-smoke-readme-deploy`; put the smoke line (`smoke: ok
|
||
(stream spread N ms)`) in the Notes column.
|
||
|
||
```sh
|
||
git add tools/smoke.sh README.md deploy/crossbar.service docs/implementer-log.md
|
||
git commit
|
||
```
|
||
|
||
## Done when
|
||
|
||
- `make smoke` prints `smoke: ok (…)`; `make gate` prints `gate: ok`; `README.md` has the seven
|
||
sections; `cmp tools/smoke.sh docs/plans/v0/_files/tools/smoke.sh` prints nothing.
|
||
|
||
## Stop and report if
|
||
|
||
- `make smoke` fails twice in the same way.
|