Alienware Aurora R11 (i7-10700, 64 GB, one RTX 3060 12 GB today; second card
when its fans are replaced). Headless, Wi-Fi only, unencrypted root by
decision (always-on box that must come back from a power cut unattended).
- llama.cpp router (CUDA, from unstable) on :11434 with the three helper
models resident (--models-max 3): ornith 9B for the Hermes approval guardian
/ title / web_extract and Honcho deriver, qwen3-4b for Honcho dialectic,
Qwen3-Embedding-0.6B for Honcho embeddings. Vision (qwen3-vl-8b) on its own
on-demand server on :11433 since it does not fit beside them in 12 GB.
- Both servers tailnet-only (tailscale0 firewall); GPUs capped at 140 W
(thermal hour 2026-09-21: 75 C plateau, no throttling).
- Model weights on the USB cache stick (ext4 label dixie-cache) bind-mounted
into /var/lib/llama-server/models; filled from straylight with dixie-push.
- 6.12 LTS kernel (mt7921u for the AX9L USB Wi-Fi arriving 2026-09-24).
- SSH keys baked in: no console after install to add them from.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
As Hermes' auxiliary model it received requests with no max_tokens and a
generation ran to 46k tokens (~70 min) after the client's 60 s timeout,
starving the 35B. Cap thinking (reasoning-budget 2048) and total output
(n-predict 4096) server-side; preset verified against a throwaway router.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Hermes resolves tirith from PATH before its self-downloaded
~/.hermes/bin/tirith. Taken from nixpkgs-unstable (0.4.1); nixos-26.05
only has 0.3.3.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
--cache-ram is per resident model, so the 8 GiB default meant up to 24 GiB
with --models-max 3. Default it to 2 GiB and give ornith-1.5-35b-a3b 16 GiB:
worst case drops to 20 GiB while the model that thrashes gets twice the cache.
ornith also goes from 2 slots of 131072 to 4 slots over one unified 262144
pool: same GPU memory, twice as many live prefixes.
Measured over 30 h before this change: 235 cache evictions and ~196 minutes
spent re-processing prompts over 20k tokens.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
models-max 3 pulled all three residents into RAM and the OOM killer
took the desktop (and Hermes) mid-switch. models-max 2 is the safe
baseline; co-residency of all three is a future-optimization, not a
today one.
Three residents (Laguna ~64G + ornith-1.5-35b ~21G + qwen3-vl-8b ~6G)
fit the ~104G TTM window with KV headroom. Needed now that both Hermes
hosts run their main loops off the router: straylight (laguna-s-2.1)
and talos (ornith-1.5-35b-a3b) each want a resident slot, leaving the
VLM as the third.
Working-tree WIP (nix-ld libs for hermes desktop) intentionally left
uncommitted.
kyleandClaude Fable 5.1