kyle/imladris
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

dixie: 9B to 6 slots over a 64k pool

Browse Source 
Room for the deriver, dialectic prefetches from three agents and their approval
guardians at once. Under full load (deriver + dialectic + guardian probes):
guardian p50 0.4 s / max 10 s, no KV-space failures. VRAM 8.9 GB.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-22 15:37:46 -07:00
co-authored by Claude Opus 5.5
parent ba1ca9387b
commit 8a6d0039d3
1 changed files with 3 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hw/dixie/default.nix
+3 -3
View File
@@ -41,10 +41,10 @@ let
[ornith-1.5-9b-uncensored] [ornith-1.5-9b-uncensored]
model = ${modelsDir}/ornith-1.5-9b-uncensored/Ornith-1.5-9B-uncensored.Q4_K_M.gguf model = ${modelsDir}/ornith-1.5-9b-uncensored/Ornith-1.5-9B-uncensored.Q4_K_M.gguf
; serves the deriver, all dialectic levels and three agents' approval guardians at once: ; serves the deriver, all dialectic levels and three agents' approval guardians at once:
; 4 slots over one 48k pool (at 2 x 16k, concurrent deriver + dialectic hit "failed to find ; 6 slots over one 64k pool (at 2 x 16k, concurrent deriver + dialectic hit "failed to find
; free space in the KV cache" and stalled) ; free space in the KV cache" and stalled)
ctx-size = 49152 ctx-size = 65536
parallel = 4 parallel = 6
; Runaway guard carried over from straylight: helper tasks need short answers. ; Runaway guard carried over from straylight: helper tasks need short answers.
reasoning-budget = 2048 reasoning-budget = 2048
n-predict = 4096 n-predict = 4096