kyle/imladris
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

straylight: llama presets for the Primer's chapter 50 student models

Browse Source 
straylight-student serves Qwen3-4B at Q8_0; straylight-student-tuned serves
the same base with the LoRA adapter trained on dixie on 2026-10-26. Both
read from /var/lib/llama-server/models/aec-ch50 with one 16k slot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-11-01 13:35:35 -08:00
co-authored by Claude Opus 5.5
parent 89c6ac4750
commit 898c8269cf
1 changed files with 15 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hw/straylight/default.nix
+15
View File
@@ -166,6 +166,21 @@ let
parallel = 1 parallel = 1
ctx-size = 32768 ctx-size = 32768
dedup-cache-models = true dedup-cache-models = true
; The Primer, chapter 50: Qwen3-4B at Q8_0 (converted from the HF checkpoint
; at 1cfa9a72) as the student, and the same base with the LoRA adapter trained
; on dixie (2026-10-26) from the 35B's tool-use traces. The before/after eval
; compares the two. One 16k slot each: the suite's turns are short.
[straylight-student]
model = ${llamaModelsDir}/aec-ch50/qwen3-4b-Q8_0.gguf
parallel = 1
ctx-size = 16384
[straylight-student-tuned]
model = ${llamaModelsDir}/aec-ch50/qwen3-4b-Q8_0.gguf
lora = ${llamaModelsDir}/aec-ch50/adapter.gguf
parallel = 1
ctx-size = 16384
''; '';
# `llama-models` shows what the router has resident; `llama-unload` frees every loaded # `llama-models` shows what the router has resident; `llama-unload` frees every loaded
# model (e.g. before a gaming session), or just the ones named on the command line. # model (e.g. before a gaming session), or just the ones named on the command line.