kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Brief P5: settle on chat-completions with server-side tool parsing

Browse Source 
Approved 2026-09-17. Evidence is in docs/decisions.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-17 00:38:32 -07:00
co-authored by Claude Fable 5.1
parent 367573b11e
commit e353cd57fa
2 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/decisions.md
+1 -1
View File
@@ -29,6 +29,7 @@ Approved by the owner on 2026-09-17. One commit each.
| P2 | Inference contract 2: the session log stores assistant messages exactly as returned, including `reasoning_content`, and replays them unchanged. Remove the "known risk" about dropped thinking blocks. | M0 (e): Ornith's template keeps every think block. | | P2 | Inference contract 2: the session log stores assistant messages exactly as returned, including `reasoning_content`, and replays them unchanged. Remove the "known risk" about dropped thinking blocks. | M0 (e): Ornith's template keeps every think block. |
| P3 | Inference contract 7: requests set `return_progress: true`; progress events count as liveness. | M0 (i): otherwise the stream is silent during prefill. | | P3 | Inference contract 7: requests set `return_progress: true`; progress events count as liveness. | M0 (i): otherwise the stream is silent during prefill. |
| P4 | Inference contract 8: the thinking cap uses `reasoning_control` and the control endpoint. | README b10809. Not yet exercised. | | P4 | Inference contract 8: the thinking cap uses `reasoning_control` and the control endpoint. | README b10809. Not yet exercised. |
| P5 | Settle the open question: chat-completions with server-side tool parsing. | M0 (b), (c). |
## Proposed changes to the design brief ## Proposed changes to the design brief
@@ -36,7 +37,6 @@ From M0 and the kickoff review. None is applied yet. Each lands as its own commi
| # | Change | Evidence | | # | Change | Evidence |
|---|---|---| |---|---|---|
| P5 | Settle the open question: chat-completions with server-side tool parsing. | M0 (b), (c). |
| P6 | Code constraints: `Decision` lives in `brokerd`, has a private field and does not implement `Deserialize`. `proto` carries a plain `DecisionRecord` for the audit log. | Rust privacy is per crate, and a deserializable type can be built by anyone. | | P6 | Code constraints: `Decision` lives in `brokerd`, has a private field and does not implement `Deserialize`. `proto` carries a plain `DecisionRecord` for the audit log. | Rust privacy is per crate, and a deserializable type can be built by anyone. |
| P7 | Inference contract 1: the baseline budget test needs the server's tokenizer, so `make gate` has an offline part and an on-device part (`make verify-device`). | `/tokenize` is a server endpoint. | | P7 | Inference contract 1: the baseline budget test needs the server's tokenizer, so `make gate` has an offline part and an on-device part (`make verify-device`). | `/tokenize` is a server endpoint. |
| P8 | Target environment: describe the shared router as deployed (router mode, `--models-max 2`, other clients on the same Ornith instance, q8_0 KV, six-hour idle unload). Drop "memory is abundant". | `docs/inference-contract.md`, "What is running". | | P8 | Target environment: describe the shared router as deployed (router mode, `--models-max 2`, other clients on the same Ornith instance, q8_0 KV, six-hour idle unload). Drop "memory is abundant". | `docs/inference-contract.md`, "What is running". |
docs/design.md
+3 -3
View File
@@ -114,9 +114,9 @@ ordinary outbound networking can reach the whole tailnet and the internet as the
10. **Serving settings are part of the repo.** Launch flags, chat template hash, sampling settings 10. **Serving settings are part of the repo.** Launch flags, chat template hash, sampling settings
(temp 0.6, top-p 0.95, top-k 20), f16 KV cache, no speculative decoding. (temp 0.6, top-p 0.95, top-k 20), f16 KV cache, no speculative decoding.
Open question settled by measurement in M0: use the server's chat-completions endpoint with Settled by M0 (`docs/inference-contract.md`): `loopd` uses the server's chat-completions endpoint
server-side tool parsing, or render the template in-process for byte-exact prefix control. Default with server-side tool parsing. It does not render the template in-process. Cache reuse held across
to chat-completions. Switch only if the M0 cache measurements fail. a multi-turn tool conversation and tool parsing had no failures in 20 trials.
## Authority contract ## Authority contract