Room for the deriver, dialectic prefetches from three agents and their approval
guardians at once. Under full load (deriver + dialectic + guardian probes):
guardian p50 0.4 s / max 10 s, no KV-space failures. VRAM 8.9 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Dialectic benchmark (8 hermes-fleet questions): Qwen3-4B answered in 2-12 s but
often wrongly (invented facts, 'no information' on things memory holds); the 9B
answered correctly. All Honcho dialectic levels now use the 9B (rift .env). Its
VRAM goes to the 9B's KV pool: at 2 x 16k, concurrent deriver + dialectic hit
'failed to find free space in the KV cache'. 9B 6.3 GB + embed 2.2 GB = 8.4 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stock Ornith template raises 'No user query found in messages.' when a request
has no plain user turn, which Honcho's dialectic tool loop sends on later
iterations: 6 of 8 'low'-level dialectic calls failed with HTTP 500 when routed to
the 9B. The patched copy treats the last message as the query instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
kv-unified lets one request use a model's whole 16k (split slots capped each at
8k and every Honcho dialectic prompt of 11-17k tokens failed). 9B gets temp 0.6 /
top-p 0.95 / top-k 20 / min-p 0: in a 20-batch replay of Honcho's deriver the
0.8 default dropped 7 batches (empty or wrong-shape output); with these, 3,
and 1 with a format instruction on rift. VRAM unchanged at 11.5 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
8192/slot (previous commit) cost 6.6 GB of compute buffers and the 9B then
failed to load; 4096/slot did not fit beside the 9B and dialectic either.
2048/slot with batch-size set explicitly; long inputs are handled on rift
instead (EMBEDDING_MAX_INPUT_TOKENS=1536 makes Honcho chunk). All three
models resident at 11.5 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
batch-size defaulted to 2048 (capping ubatch) and ctx 8192 over 4 slots left
2048 per slot, so Honcho inputs over 2048 tokens failed with 'input is too
large to process'. Now batch/ubatch 8192, ctx 16384 over 2 slots = 8192/slot,
matching Honcho's EMBEDDING.MAX_INPUT_TOKENS. Verified a 6191-token input.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Alienware Aurora R11 (i7-10700, 64 GB, one RTX 3060 12 GB today; second card
when its fans are replaced). Headless, Wi-Fi only, unencrypted root by
decision (always-on box that must come back from a power cut unattended).
- llama.cpp router (CUDA, from unstable) on :11434 with the three helper
models resident (--models-max 3): ornith 9B for the Hermes approval guardian
/ title / web_extract and Honcho deriver, qwen3-4b for Honcho dialectic,
Qwen3-Embedding-0.6B for Honcho embeddings. Vision (qwen3-vl-8b) on its own
on-demand server on :11433 since it does not fit beside them in 12 GB.
- Both servers tailnet-only (tailscale0 firewall); GPUs capped at 140 W
(thermal hour 2026-09-21: 75 C plateau, no throttling).
- Model weights on the USB cache stick (ext4 label dixie-cache) bind-mounted
into /var/lib/llama-server/models; filled from straylight with dixie-push.
- 6.12 LTS kernel (mt7921u for the AX9L USB Wi-Fi arriving 2026-09-24).
- SSH keys baked in: no console after install to add them from.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
As Hermes' auxiliary model it received requests with no max_tokens and a
generation ran to 46k tokens (~70 min) after the client's 60 s timeout,
starving the 35B. Cap thinking (reasoning-budget 2048) and total output
(n-predict 4096) server-side; preset verified against a throwaway router.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Hermes resolves tirith from PATH before its self-downloaded
~/.hermes/bin/tirith. Taken from nixpkgs-unstable (0.4.1); nixos-26.05
only has 0.3.3.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
--cache-ram is per resident model, so the 8 GiB default meant up to 24 GiB
with --models-max 3. Default it to 2 GiB and give ornith-1.5-35b-a3b 16 GiB:
worst case drops to 20 GiB while the model that thrashes gets twice the cache.
ornith also goes from 2 slots of 131072 to 4 slots over one unified 262144
pool: same GPU memory, twice as many live prefixes.
Measured over 30 h before this change: 235 cache evictions and ~196 minutes
spent re-processing prompts over 20k tokens.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
models-max 3 pulled all three residents into RAM and the OOM killer
took the desktop (and Hermes) mid-switch. models-max 2 is the safe
baseline; co-residency of all three is a future-optimization, not a
today one.
Three residents (Laguna ~64G + ornith-1.5-35b ~21G + qwen3-vl-8b ~6G)
fit the ~104G TTM window with KV headroom. Needed now that both Hermes
hosts run their main loops off the router: straylight (laguna-s-2.1)
and talos (ornith-1.5-35b-a3b) each want a resident slot, leaving the
VLM as the third.
Working-tree WIP (nix-ld libs for hermes desktop) intentionally left
uncommitted.
Official laguna-s-2.1-DFlash-BF16.gguf is on disk but this llama.cpp
rejects it (expected 76 tensors, got 69). Leave the draft unwired so
Laguna loads; keep amdgpu DPM high.
A second request (opencode background agent, pi alongside opencode) no
longer queues behind the first. kv-unified keeps the full 131072-token
context available to either slot instead of splitting it to 65536.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The unit re-applied --https=443 every 5s and fought Mattermost for
that port. llama is served on :10000, managed outside Nix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OpenCode talks to https://straylight.scylla-hammerhead.ts.net/v1, which is
Tailscale Serve on :443 proxying localhost:11434. Re-apply the serve
config on a timer so cannot leave the API dark.
whisper.cpp large-v3-turbo on :11435 (CPU, OpenAI transcriptions path)
so ASR does not take GTT from llama-server. Flash-Next is the cygnal
IQ4_XS-NGQ4 GGUF (~98 GB, gfx1151), qwen4exp, mmproj pinned; unload
Laguna before loading.
Managed /etc/opencode/opencode.json points at straylight:11434 with
the router model ids, Laguna as default, Ornith 1.5 9B as small_model,
and a vision subagent on Qwen3-VL-8B abliterated.
Ornith 1.5 9B is a dense text model, not vision. Replace that role with
prithivMLmods Qwen3-VL-8B Instruct abliterated v2 (mradermacher Q4_K_M)
and pin mmproj-url so router mode actually loads the projector.
Bind 0.0.0.0 and open 11434 on tailscale0 only, so localhost and
other tailnet machines can reach the OpenAI-compatible API. LAN
stays closed. rebuild-nixos deploys committed master, not the
working tree.
Drop the b9828 source pin and take llama.cpp from a refreshed
nixpkgs-unstable (0.4.0), which supports the Laguna and Gemma 4
architectures. Run llama-server in router mode with a preset file:
abliterated Qwen3.6-35B-A3B and Gemma 4 26B-A4B for benchmarking,
Poolside Laguna S 2.1 as the primary coding agent, and Ornith kept for
comparison. At most two models stay resident; idle models unload after
six hours. Add llama-models and llama-unload helper commands.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The TTM default caps GTT at half of RAM (~62 GiB), which is too small for
the 118B-parameter model straylight will serve. Set ttm.pages_limit and
ttm.page_pool_size to 27262976 pages (104 GiB); takes effect on reboot.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pin herdr v0.9.0 from the upstream flake (nixpkgs-unstable only has
0.7.1) and install it on straylight to host coding-agent terminals.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Update for the 26.05 bump and host changes: 8 hosts, orion is now a
server, straylight/svc added, vade on desktop-light, MCP modules,
and the current LUKS coverage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BIOS boot with GRUB on /dev/xvda, MCP agent via systemd,
mc-proxy and MCNS as containers via MCP agent.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The LUKS device is named "luks-5c5e94fc-..." in hardware-configuration.nix
which already has the FIDO2 options. The "crypted" reference caused a build
error. Also fix duplicate attribute definitions and unnecessary config wrapper.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New root CA cert issued during Metacrypt vault rebuild. Same key
usage (Certificate Sign, CRL Sign), new validity period (2026-2046).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Same key, added CRL Sign to key usage extensions. Distributed to
all nodes and NixOS system trust store.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use nixpkgs.hostPlatform module instead of deprecated system arg to lib.nixosSystem
- Rename services.logind.powerKey to services.logind.settings.Login.HandlePowerKey
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rootless podman deeply caches the UID in storage, subuid mappings,
and systemd sessions. Changing it destroys all container state.
Reference: log/2026-04-03-uid-incident.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All nodes now list 1.1.1.1 and 8.8.8.8 as fallback nameservers after
MCNS. When MCNS is down, internal names (.svc.mcp.metacircular.net)
fail but external DNS (google.com, github.com, etc.) keeps working.
Lesson from 2026-04-03 incident: without fallbacks, MCNS failure
caused total DNS blackout including external services, forcing
Tailscale to be disabled to restore any DNS resolution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
UID 995 conflicted with sshd on orion. Pin to 850 (the 800-899 range
is unused on all nodes and well below NixOS auto-assign range).
Pin GID to 850 as well for consistency.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
UID 995 conflicted with sshd on orion. Let NixOS auto-assign the UID
for the mcp system user. Use systemd's %U specifier for XDG_RUNTIME_DIR
instead of the hardcoded UID.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The FIDO2 crypttab options are already on the correct UUID-named device
in hardware-configuration.nix; the "crypted" name only applies to
disko-provisioned hosts (rift).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The module used explicit `config = { ... }` but also had duplicate
networking.nameservers and services.resolved.domains at the top level,
causing a NixOS module evaluation error. Merged the Tailscale nameserver
into the config block and removed the duplicates.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
kyleandClaude Opus 5.5
