kyle/imladris
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

dixie: 9B to 8 slots over a 96k pool

Browse Source 
A Honcho catch-up wave (48 deriver batches + 11 dreams + dialectic prefetches)
held all 6 slots with long thinking generations; tiny aux calls (title,
approval guardian) queued behind them and title_generation timed out at 60 s.
Two spare slots plus a bigger shared KV pool (so 6 long prompts can't exhaust
it). VRAM 9.7 GB.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-22 19:14:45 -07:00
co-authored by Claude Opus 5.5
parent f28c1ae5f0
commit b743d3f360
1 changed files with 4 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hw/dixie/default.nix
+4 -3
View File
@@ -41,10 +41,11 @@ let
[ornith-1.5-9b-uncensored] [ornith-1.5-9b-uncensored]
model = ${modelsDir}/ornith-1.5-9b-uncensored/Ornith-1.5-9B-uncensored.Q4_K_M.gguf model = ${modelsDir}/ornith-1.5-9b-uncensored/Ornith-1.5-9B-uncensored.Q4_K_M.gguf
; serves the deriver, all dialectic levels and three agents' approval guardians at once: ; serves the deriver, all dialectic levels and three agents' approval guardians at once:
; 6 slots over one 64k pool (at 2 x 16k, concurrent deriver + dialectic hit "failed to find ; 8 slots over one 96k pool so tiny guardian/title calls still find a slot while Honcho
; catch-up (deriver + dreams + dialectic) holds 6 (at 2 x 16k, concurrent calls hit "failed to find
; free space in the KV cache" and stalled) ; free space in the KV cache" and stalled)
ctx-size = 65536 ctx-size = 98304
parallel = 6 parallel = 8
; Runaway guard carried over from straylight: helper tasks need short answers. ; Runaway guard carried over from straylight: helper tasks need short answers.
reasoning-budget = 2048 reasoning-budget = 2048
n-predict = 4096 n-predict = 4096