Brief P11: serving settings are recorded here and owned by the NixOS config
Approved 2026-09-17. Evidence is in docs/decisions.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
+6
-2
@@ -125,8 +125,12 @@ ordinary outbound networking can reach the whole tailnet and the internet as the
|
||||
9. **Startup self-test.** On boot `loopd` checks: tool-call round trip parses, turn-2 prompt
|
||||
processing count shows a cache hit, configured context matches what the server reports.
|
||||
It refuses to start if any check fails.
|
||||
10. **Serving settings are part of the repo.** Launch flags, chat template hash, sampling settings
|
||||
(temp 0.6, top-p 0.95, top-k 20), f16 KV cache, no speculative decoding.
|
||||
10. **Serving settings are recorded, not owned.** The launch flags live in the owner's NixOS
|
||||
configuration (`~/src/nixos` on straylight). This repo records the expected values: chat
|
||||
template hash, per-slot context, slot count, q8_0 KV cache, no speculative decoding. The
|
||||
startup self-test compares them with `/props` and `/slots`. Sampling settings (temp 0.6,
|
||||
top-p 0.95, top-k 20) are sent with every request, because the server's default temperature
|
||||
is 1.0.
|
||||
|
||||
Settled by M0 (`docs/inference-contract.md`): `loopd` uses the server's chat-completions endpoint
|
||||
with server-side tool parsing. It does not render the template in-process. Cache reuse held across
|
||||
|
||||
Reference in New Issue
Block a user