kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record shared-router decision and the brief changes it implies

Browse Source 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-17 00:27:53 -07:00
co-authored by Claude Fable 5.1
parent cbdaaa0d50
commit 5bd0444aeb
1 changed files with 6 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/decisions.md
+6 -2
View File
@@ -6,6 +6,7 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit
| Date | Decision | Reason |
|---|---|---|
| 2026-09-17 | Boxmaker uses the shared `llama-server` router and the shared Ornith instance. No dedicated instance. `inferproxy` stays. | The owner runs coding agents against the same router, and a second resident copy of Ornith does not fit beside Laguna. |
| 2026-09-17 | The harness runs on straylight, the same host as `llama-server`. | Owner's choice. One host to secure, and no inference traffic crosses the tailnet. |
| 2026-09-17 | Container runtime is rootless Podman. | It is already the standard runtime on straylight and the owner's other hosts. |
| 2026-09-17 | Implementation is done by Laguna S 2.1 through OpenCode on straylight. Design, specs, measurement and review are done by a stronger model. Ornith-1.5-35B-A3B remains the model the harness serves. | The owner wants to test a local model on real implementation work. Plans must be written as small closed tasks with tests specified up front. |
@@ -15,7 +16,6 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit
| Decision | Needed by |
|---|---|
| Serving setup: a dedicated `llama-server` unit for Boxmaker on a Unix socket (recommended in `docs/inference-contract.md`), or the shared router plus `inferproxy`. Decides whether the `inferproxy` crate exists and whether three pinned slots are available. | M1 |
| Secret store backend, and where the v0 file's key lives. straylight has no secrets manager today. | M3 |
| Whether cloud-led sessions are ever allowed, and for which data classes. | M6 |
@@ -32,4 +32,8 @@ From M0 and the kickoff review. None is applied yet. Each lands as its own commi
| P5 | Settle the open question: chat-completions with server-side tool parsing. | M0 (b), (c). |
| P6 | Code constraints: `Decision` lives in `brokerd`, has a private field and does not implement `Deserialize`. `proto` carries a plain `DecisionRecord` for the audit log. | Rust privacy is per crate, and a deserializable type can be built by anyone. |
| P7 | Inference contract 1: the baseline budget test needs the server's tokenizer, so `make gate` has an offline part and an on-device part (`make verify-device`). | `/tokenize` is a server endpoint. |
| P8 | Target environment: replace "memory is abundant" and the f16 and slot assumptions with the measured setup, once the serving decision is made. | `docs/inference-contract.md`, "What is running". |
| P8 | Target environment: describe the shared router as deployed (router mode, `--models-max 2`, other clients on the same Ornith instance, q8_0 KV, six-hour idle unload). Drop "memory is abundant". | `docs/inference-contract.md`, "What is running". |
| P9 | Inference contract 5: slots are pinned with `id_slot` but not reserved. Another client can evict a harness slot, and the router can unload Ornith. Cache loss is an expected event: `loopd` detects it (`cache_n` far below the previous request's total), logs it in the session log, and carries on. It is never an error. | Shared router decision. |
| P10 | Inference contract 7: a pinned request can queue behind another client's request on the same slot, and the router can spend tens of seconds reloading the model. The liveness timer starts at the first byte received, and a separate, longer "waiting for slot or model" limit covers the time before it. | Shared router decision. Needs a measurement of what the stream sends while queued. |
| P11 | Inference contract 10: the serving flags live in `~/src/nixos`, not here. This repo records the expected values (template hash, per-slot `n_ctx`, slot count, KV type, sampling) and the startup self-test compares them with `/props` and `/slots`. KV type is recorded as q8_0. | Shared router decision. |
| P12 | Network isolation 2: `inferproxy` is kept. Remove the conditional about dropping it. Its upstream is the router's TCP listener on the host. | Shared router decision; M0 (g). |