From 72cae29d5c4e99371fcafd2009f2421e58c62caf Mon Sep 17 00:00:00 2001 From: "K. Isom" Date: Tue, 22 Sep 2026 12:56:16 -0700 Subject: [PATCH] dixie: unified KV per model; Qwen thinking sampling for the 9B kv-unified lets one request use a model's whole 16k (split slots capped each at 8k and every Honcho dialectic prompt of 11-17k tokens failed). 9B gets temp 0.6 / top-p 0.95 / top-k 20 / min-p 0: in a 20-batch replay of Honcho's deriver the 0.8 default dropped 7 batches (empty or wrong-shape output); with these, 3, and 1 with a format instruction on rift. VRAM unchanged at 11.5 GB. Co-Authored-By: Claude Opus 5.5 (1M context) --- hw/dixie/default.nix | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/hw/dixie/default.nix b/hw/dixie/default.nix index 7a21a63..7057ef9 100644 --- a/hw/dixie/default.nix +++ b/hw/dixie/default.nix @@ -31,6 +31,9 @@ let ; three resident models share 12 GB: keep contexts modest ctx-size = 16384 parallel = 2 + ; one KV pool per model shared by its slots, so a single long request can use all 16k + ; (split slots capped each at 8k: Honcho dialectic prompts of 11-17k all failed) + kv-unified = true cache-ram = 512 ; Ornith 1.5 9B, refusal-ablated. Hermes auxiliary model (approval guardian, title, @@ -40,6 +43,12 @@ let ; Runaway guard carried over from straylight: helper tasks need short answers. reasoning-budget = 2048 n-predict = 4096 + ; Qwen thinking-mode sampling (llama-server's 0.8 default made the Honcho deriver drop + ; ~1 in 3 batches, measured 2026-09-22). Clients that send their own temperature win. + temp = 0.6 + top-p = 0.95 + top-k = 20 + min-p = 0 ; Honcho dialectic: non-thinking instruct model so answers are short and fast. [honcho-dialectic]