Alienware Aurora R11 (i7-10700, 64 GB, one RTX 3060 12 GB today; second card
when its fans are replaced). Headless, Wi-Fi only, unencrypted root by
decision (always-on box that must come back from a power cut unattended).
- llama.cpp router (CUDA, from unstable) on :11434 with the three helper
models resident (--models-max 3): ornith 9B for the Hermes approval guardian
/ title / web_extract and Honcho deriver, qwen3-4b for Honcho dialectic,
Qwen3-Embedding-0.6B for Honcho embeddings. Vision (qwen3-vl-8b) on its own
on-demand server on :11433 since it does not fit beside them in 12 GB.
- Both servers tailnet-only (tailscale0 firewall); GPUs capped at 140 W
(thermal hour 2026-09-21: 75 C plateau, no throttling).
- Model weights on the USB cache stick (ext4 label dixie-cache) bind-mounted
into /var/lib/llama-server/models; filled from straylight with dixie-push.
- 6.12 LTS kernel (mt7921u for the AX9L USB Wi-Fi arriving 2026-09-24).
- SSH keys baked in: no console after install to add them from.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
As Hermes' auxiliary model it received requests with no max_tokens and a
generation ran to 46k tokens (~70 min) after the client's 60 s timeout,
starving the 35B. Cap thinking (reasoning-budget 2048) and total output
(n-predict 4096) server-side; preset verified against a throwaway router.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Hermes resolves tirith from PATH before its self-downloaded
~/.hermes/bin/tirith. Taken from nixpkgs-unstable (0.4.1); nixos-26.05
only has 0.3.3.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
--cache-ram is per resident model, so the 8 GiB default meant up to 24 GiB
with --models-max 3. Default it to 2 GiB and give ornith-1.5-35b-a3b 16 GiB:
worst case drops to 20 GiB while the model that thrashes gets twice the cache.
ornith also goes from 2 slots of 131072 to 4 slots over one unified 262144
pool: same GPU memory, twice as many live prefixes.
Measured over 30 h before this change: 235 cache evictions and ~196 minutes
spent re-processing prompts over 20k tokens.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
models-max 3 pulled all three residents into RAM and the OOM killer
took the desktop (and Hermes) mid-switch. models-max 2 is the safe
baseline; co-residency of all three is a future-optimization, not a
today one.
Three residents (Laguna ~64G + ornith-1.5-35b ~21G + qwen3-vl-8b ~6G)
fit the ~104G TTM window with KV headroom. Needed now that both Hermes
hosts run their main loops off the router: straylight (laguna-s-2.1)
and talos (ornith-1.5-35b-a3b) each want a resident slot, leaving the
VLM as the third.
Working-tree WIP (nix-ld libs for hermes desktop) intentionally left
uncommitted.
Official laguna-s-2.1-DFlash-BF16.gguf is on disk but this llama.cpp
rejects it (expected 76 tensors, got 69). Leave the draft unwired so
Laguna loads; keep amdgpu DPM high.
A second request (opencode background agent, pi alongside opencode) no
longer queues behind the first. kv-unified keeps the full 131072-token
context available to either slot instead of splitting it to 65536.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The unit re-applied --https=443 every 5s and fought Mattermost for
that port. llama is served on :10000, managed outside Nix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OpenCode talks to https://straylight.scylla-hammerhead.ts.net/v1, which is
Tailscale Serve on :443 proxying localhost:11434. Re-apply the serve
config on a timer so cannot leave the API dark.
whisper.cpp large-v3-turbo on :11435 (CPU, OpenAI transcriptions path)
so ASR does not take GTT from llama-server. Flash-Next is the cygnal
IQ4_XS-NGQ4 GGUF (~98 GB, gfx1151), qwen4exp, mmproj pinned; unload
Laguna before loading.
Managed /etc/opencode/opencode.json points at straylight:11434 with
the router model ids, Laguna as default, Ornith 1.5 9B as small_model,
and a vision subagent on Qwen3-VL-8B abliterated.
Ornith 1.5 9B is a dense text model, not vision. Replace that role with
prithivMLmods Qwen3-VL-8B Instruct abliterated v2 (mradermacher Q4_K_M)
and pin mmproj-url so router mode actually loads the projector.
Bind 0.0.0.0 and open 11434 on tailscale0 only, so localhost and
other tailnet machines can reach the OpenAI-compatible API. LAN
stays closed. rebuild-nixos deploys committed master, not the
working tree.
Drop the b9828 source pin and take llama.cpp from a refreshed
nixpkgs-unstable (0.4.0), which supports the Laguna and Gemma 4
architectures. Run llama-server in router mode with a preset file:
abliterated Qwen3.6-35B-A3B and Gemma 4 26B-A4B for benchmarking,
Poolside Laguna S 2.1 as the primary coding agent, and Ornith kept for
comparison. At most two models stay resident; idle models unload after
six hours. Add llama-models and llama-unload helper commands.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The TTM default caps GTT at half of RAM (~62 GiB), which is too small for
the 118B-parameter model straylight will serve. Set ttm.pages_limit and
ttm.page_pool_size to 27262976 pages (104 GiB); takes effect on reboot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pin herdr v0.9.0 from the upstream flake (nixpkgs-unstable only has
0.7.1) and install it on straylight to host coding-agent terminals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Update for the 26.05 bump and host changes: 8 hosts, orion is now a
server, straylight/svc added, vade on desktop-light, MCP modules,
and the current LUKS coverage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kyleandClaude Fable 5.1
