Brief P11: serving settings are recorded here and owned by the NixOS config

Approved 2026-09-17. Evidence is in docs/decisions.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-09-17 00:38:32 -07:00
co-authored by Claude Fable 5.1
parent adab43add1
commit a4ebdc11af
2 changed files with 7 additions and 3 deletions
+6 -2
View File
@@ -125,8 +125,12 @@ ordinary outbound networking can reach the whole tailnet and the internet as the
9. **Startup self-test.** On boot `loopd` checks: tool-call round trip parses, turn-2 prompt
processing count shows a cache hit, configured context matches what the server reports.
It refuses to start if any check fails.
10. **Serving settings are part of the repo.** Launch flags, chat template hash, sampling settings
(temp 0.6, top-p 0.95, top-k 20), f16 KV cache, no speculative decoding.
10. **Serving settings are recorded, not owned.** The launch flags live in the owner's NixOS
configuration (`~/src/nixos` on straylight). This repo records the expected values: chat
template hash, per-slot context, slot count, q8_0 KV cache, no speculative decoding. The
startup self-test compares them with `/props` and `/slots`. Sampling settings (temp 0.6,
top-p 0.95, top-k 20) are sent with every request, because the server's default temperature
is 1.0.
Settled by M0 (`docs/inference-contract.md`): `loopd` uses the server's chat-completions endpoint
with server-side tool parsing. It does not render the template in-process. Cache reuse held across