kyleandClaude Fable 5.1
06c05e0023
spike/m0.py is throwaway. Findings are in docs/inference-contract.md: cache reuse and tool parsing pass through chat-completions, the tools array must stay fixed per epoch, and the shared router cannot meet the slot contract. Clean throughput and slot pinning are still open because another session was using the GPU. docs/decisions.md lists the brief changes this implies as proposals. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
3.2 KiB
3.2 KiB
Decisions
Newest first. A decision that changes docs/design.md lands in the same commit as the change.
Decided
| Date | Decision | Reason |
|---|---|---|
| 2026-09-17 | The harness runs on straylight, the same host as llama-server. |
Owner's choice. One host to secure, and no inference traffic crosses the tailnet. |
| 2026-09-17 | Container runtime is rootless Podman. | It is already the standard runtime on straylight and the owner's other hosts. |
| 2026-09-17 | Implementation is done by Laguna S 2.1 through OpenCode on straylight. Design, specs, measurement and review are done by a stronger model. Ornith-1.5-35B-A3B remains the model the harness serves. | The owner wants to test a local model on real implementation work. Plans must be written as small closed tasks with tests specified up front. |
| 2026-09-17 | M0 is run by the design model, not by Laguna. | M0 is measurement and interpretation, and its findings bind the design. |
Open
| Decision | Needed by |
|---|---|
Serving setup: a dedicated llama-server unit for Boxmaker on a Unix socket (recommended in docs/inference-contract.md), or the shared router plus inferproxy. Decides whether the inferproxy crate exists and whether three pinned slots are available. |
M1 |
| Secret store backend, and where the v0 file's key lives. straylight has no secrets manager today. | M3 |
| Whether cloud-led sessions are ever allowed, and for which data classes. | M6 |
Proposed changes to the design brief
From M0 and the kickoff review. None is applied yet. Each lands as its own commit once the owner agrees.
| # | Change | Evidence |
|---|---|---|
| P1 | Inference contract 6: the tools array is fixed per epoch. find_tool returns schemas as a tool result and the model calls them through a call_tool(name, arguments) meta-tool. Never instruct the model to call an undeclared tool. |
M0 (h): adding a tool re-read the whole prompt; undeclared calls were coerced into write_file. |
| P2 | Inference contract 2: the session log stores assistant messages exactly as returned, including reasoning_content, and replays them unchanged. Remove the "known risk" about dropped thinking blocks. |
M0 (e): Ornith's template keeps every think block. |
| P3 | Inference contract 7: requests set return_progress: true; progress events count as liveness. |
M0 (i): otherwise the stream is silent during prefill. |
| P4 | Inference contract 8: the thinking cap uses reasoning_control and the control endpoint. |
README b10809. Not yet exercised. |
| P5 | Settle the open question: chat-completions with server-side tool parsing. | M0 (b), (c). |
| P6 | Code constraints: Decision lives in brokerd, has a private field and does not implement Deserialize. proto carries a plain DecisionRecord for the audit log. |
Rust privacy is per crate, and a deserializable type can be built by anyone. |
| P7 | Inference contract 1: the baseline budget test needs the server's tokenizer, so make gate has an offline part and an on-device part (make verify-device). |
/tokenize is a server endpoint. |
| P8 | Target environment: replace "memory is abundant" and the f16 and slot assumptions with the measured setup, once the serving decision is made. | docs/inference-contract.md, "What is running". |