As Hermes' auxiliary model it received requests with no max_tokens and a
generation ran to 46k tokens (~70 min) after the client's 60 s timeout,
starving the 35B. Cap thinking (reasoning-budget 2048) and total output
(n-predict 4096) server-side; preset verified against a throwaway router.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Hermes resolves tirith from PATH before its self-downloaded
~/.hermes/bin/tirith. Taken from nixpkgs-unstable (0.4.1); nixos-26.05
only has 0.3.3.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
--cache-ram is per resident model, so the 8 GiB default meant up to 24 GiB
with --models-max 3. Default it to 2 GiB and give ornith-1.5-35b-a3b 16 GiB:
worst case drops to 20 GiB while the model that thrashes gets twice the cache.
ornith also goes from 2 slots of 131072 to 4 slots over one unified 262144
pool: same GPU memory, twice as many live prefixes.
Measured over 30 h before this change: 235 cache evictions and ~196 minutes
spent re-processing prompts over 20k tokens.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
models-max 3 pulled all three residents into RAM and the OOM killer
took the desktop (and Hermes) mid-switch. models-max 2 is the safe
baseline; co-residency of all three is a future-optimization, not a
today one.
Three residents (Laguna ~64G + ornith-1.5-35b ~21G + qwen3-vl-8b ~6G)
fit the ~104G TTM window with KV headroom. Needed now that both Hermes
hosts run their main loops off the router: straylight (laguna-s-2.1)
and talos (ornith-1.5-35b-a3b) each want a resident slot, leaving the
VLM as the third.
Working-tree WIP (nix-ld libs for hermes desktop) intentionally left
uncommitted.
Official laguna-s-2.1-DFlash-BF16.gguf is on disk but this llama.cpp
rejects it (expected 76 tensors, got 69). Leave the draft unwired so
Laguna loads; keep amdgpu DPM high.
A second request (opencode background agent, pi alongside opencode) no
longer queues behind the first. kv-unified keeps the full 131072-token
context available to either slot instead of splitting it to 65536.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The unit re-applied --https=443 every 5s and fought Mattermost for
that port. llama is served on :10000, managed outside Nix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OpenCode talks to https://straylight.scylla-hammerhead.ts.net/v1, which is
Tailscale Serve on :443 proxying localhost:11434. Re-apply the serve
config on a timer so cannot leave the API dark.
whisper.cpp large-v3-turbo on :11435 (CPU, OpenAI transcriptions path)
so ASR does not take GTT from llama-server. Flash-Next is the cygnal
IQ4_XS-NGQ4 GGUF (~98 GB, gfx1151), qwen4exp, mmproj pinned; unload
Laguna before loading.
Ornith 1.5 9B is a dense text model, not vision. Replace that role with
prithivMLmods Qwen3-VL-8B Instruct abliterated v2 (mradermacher Q4_K_M)
and pin mmproj-url so router mode actually loads the projector.
Bind 0.0.0.0 and open 11434 on tailscale0 only, so localhost and
other tailnet machines can reach the OpenAI-compatible API. LAN
stays closed. rebuild-nixos deploys committed master, not the
working tree.
Drop the b9828 source pin and take llama.cpp from a refreshed
nixpkgs-unstable (0.4.0), which supports the Laguna and Gemma 4
architectures. Run llama-server in router mode with a preset file:
abliterated Qwen3.6-35B-A3B and Gemma 4 26B-A4B for benchmarking,
Poolside Laguna S 2.1 as the primary coding agent, and Ornith kept for
comparison. At most two models stay resident; idle models unload after
six hours. Add llama-models and llama-unload helper commands.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The TTM default caps GTT at half of RAM (~62 GiB), which is too small for
the 118B-parameter model straylight will serve. Set ttm.pages_limit and
ttm.page_pool_size to 27262976 pages (104 GiB); takes effect on reboot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pin herdr v0.9.0 from the upstream flake (nixpkgs-unstable only has
0.7.1) and install it on straylight to host coding-agent terminals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Host-only 10.99.0.0/24 bridge with no uplink/NAT; firewall drops VM
traffic leaving the bridge so unikernel VMs can reach only the gateway
(mc-proxy). Implements Phase 2 mandatory-mediation networking.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Override the shared mcp.nix sandbox (PrivateDevices) on straylight so the
MCP agent can boot Nanos unikernel VMs under QEMU/KVM and manage TAP
devices for isolated networking.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add configs/mcp.nix (mcp user UID 850 + mcp-agent service) and open
firewall ports for DNS/mc-proxy/agent/master as straylight takes over
the master + MCIAS + MCNS core role from rift.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All nodes now list 1.1.1.1 and 8.8.8.8 as fallback nameservers after
MCNS. When MCNS is down, internal names (.svc.mcp.metacircular.net)
fail but external DNS (google.com, github.com, etc.) keeps working.
Lesson from 2026-04-03 incident: without fallbacks, MCNS failure
caused total DNS blackout including external services, forcing
Tailscale to be disabled to restore any DNS resolution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace fragile environment.etc.crypttab.text with
boot.initrd.luks.devices for the second SSD, matching
the pattern used for the root drive.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
kyle
Kyle IsomandClaude Opus 4.8