kyle/imladris
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

dixie: 9B KV pool 96k -> 128k

Browse Source 
The unified 96k pool overflowed ~every 2-3 min under Honcho load
(39 events 2026-09-23 00:50-04:28). Each overflow fails every in-flight
slot with 'Context size has been exceeded', so Honcho retries and the
guardians' calls die with it. KV is ~20 KB/token on this model; 128k
costs ~0.65 GB of the ~2.5 GB free.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-23 04:47:24 -07:00
co-authored by Claude Opus 5.5
parent afa33ee8ee
commit 8790005c53
1 changed files with 5 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hw/dixie/default.nix
+5 -2
View File
@@ -43,8 +43,11 @@ let
; serves the deriver, all dialectic levels and three agents' approval guardians at once:
; 8 slots over one 96k pool so tiny guardian/title calls still find a slot while Honcho
; catch-up (deriver + dreams + dialectic) holds 6 (at 2 x 16k, concurrent calls hit "failed to find
; free space in the KV cache" and stalled)
ctx-size = 98304
; free space in the KV cache" and stalled). 96k overflowed ~every 2-3 min under
; load on 2026-09-23 (39 events 00:50-04:28; each fails ALL in-flight slots with
; "Context size has been exceeded"). KV is ~20 KB/token here (9B process was 7.5 GB
; at 96k incl. 5.5 GB weights), so 128k costs ~0.65 GB of the ~2.5 GB free.
ctx-size = 131072
parallel = 8
; Runaway guard carried over from straylight: helper tasks need short answers.
reasoning-budget = 2048