From 09d16f2f394b5ece90b764dd4d13a8a58b6413d1 Mon Sep 17 00:00:00 2001 From: "K. Isom" Date: Mon, 21 Sep 2026 01:13:30 -0700 Subject: [PATCH] llama-server: cap ornith 9B helper output (runaway guard) As Hermes' auxiliary model it received requests with no max_tokens and a generation ran to 46k tokens (~70 min) after the client's 60 s timeout, starving the 35B. Cap thinking (reasoning-budget 2048) and total output (n-predict 4096) server-side; preset verified against a throwaway router. Co-Authored-By: Claude Fable 5.1 --- hw/straylight/default.nix | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/straylight/default.nix b/hw/straylight/default.nix index c1849fd..83aaf03 100644 --- a/hw/straylight/default.nix +++ b/hw/straylight/default.nix @@ -71,6 +71,13 @@ let [ornith-1.5-9b-uncensored] hf-repo = mradermacher/Ornith-1.5-9B-uncensored-GGUF:Q4_K_M dedup-cache-models = true + ; Runaway guard (2026-09-21): as Hermes' auxiliary model (approval, title, web + ; extract) it got requests with no max_tokens, never converged, and one generation + ; ran to 46k tokens (~70 min at 11 t/s) after the client's 60 s timeout, starving the + ; 35B. Helper tasks need short answers: cap thinking and total output server-side. + ; A client that sends its own smaller max_tokens still wins. + reasoning-budget = 2048 + n-predict = 4096 ; Qwen3-VL-8B Instruct, refusal-ablated (prithivMLmods v2 / mradermacher). ; Dedicated VLM in the same size class as Ornith 1.5 9B. Explicit mmproj-url