Dialectic benchmark (8 hermes-fleet questions): Qwen3-4B answered in 2-12 s but
often wrongly (invented facts, 'no information' on things memory holds); the 9B
answered correctly. All Honcho dialectic levels now use the 9B (rift .env). Its
VRAM goes to the 9B's KV pool: at 2 x 16k, concurrent deriver + dialectic hit
'failed to find free space in the KV cache'. 9B 6.3 GB + embed 2.2 GB = 8.4 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stock Ornith template raises 'No user query found in messages.' when a request
has no plain user turn, which Honcho's dialectic tool loop sends on later
iterations: 6 of 8 'low'-level dialectic calls failed with HTTP 500 when routed to
the 9B. The patched copy treats the last message as the query instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
kv-unified lets one request use a model's whole 16k (split slots capped each at
8k and every Honcho dialectic prompt of 11-17k tokens failed). 9B gets temp 0.6 /
top-p 0.95 / top-k 20 / min-p 0: in a 20-batch replay of Honcho's deriver the
0.8 default dropped 7 batches (empty or wrong-shape output); with these, 3,
and 1 with a format instruction on rift. VRAM unchanged at 11.5 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
8192/slot (previous commit) cost 6.6 GB of compute buffers and the 9B then
failed to load; 4096/slot did not fit beside the 9B and dialectic either.
2048/slot with batch-size set explicitly; long inputs are handled on rift
instead (EMBEDDING_MAX_INPUT_TOKENS=1536 makes Honcho chunk). All three
models resident at 11.5 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
batch-size defaulted to 2048 (capping ubatch) and ctx 8192 over 4 slots left
2048 per slot, so Honcho inputs over 2048 tokens failed with 'input is too
large to process'. Now batch/ubatch 8192, ctx 16384 over 2 slots = 8192/slot,
matching Honcho's EMBEDDING.MAX_INPUT_TOKENS. Verified a 6191-token input.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Alienware Aurora R11 (i7-10700, 64 GB, one RTX 3060 12 GB today; second card
when its fans are replaced). Headless, Wi-Fi only, unencrypted root by
decision (always-on box that must come back from a power cut unattended).
- llama.cpp router (CUDA, from unstable) on :11434 with the three helper
models resident (--models-max 3): ornith 9B for the Hermes approval guardian
/ title / web_extract and Honcho deriver, qwen3-4b for Honcho dialectic,
Qwen3-Embedding-0.6B for Honcho embeddings. Vision (qwen3-vl-8b) on its own
on-demand server on :11433 since it does not fit beside them in 12 GB.
- Both servers tailnet-only (tailscale0 firewall); GPUs capped at 140 W
(thermal hour 2026-09-21: 75 C plateau, no throttling).
- Model weights on the USB cache stick (ext4 label dixie-cache) bind-mounted
into /var/lib/llama-server/models; filled from straylight with dixie-push.
- 6.12 LTS kernel (mt7921u for the AX9L USB Wi-Fi arriving 2026-09-24).
- SSH keys baked in: no console after install to add them from.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
kyleandClaude Opus 5.5