Qwen3-Embedding is a causal model; its card pools the last token. With cls every
input embedded as its first token's state ("the cat sat on the mat" and "the stock
market crashed today" at cosine 1.0). A CPU side server with --pooling last
reproduced the card's example scores within 0.002 without a manual EOS. Honcho's
stored vectors and sift's embed cache are re-embedded / flushed after the switch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VictoriaMetrics on 127.0.0.1:8428. LoadCredential for the Mattermost
webhook and the sift scrape token. Honcho scrapes keep a closed metric
name set. Meta-alerts for target-down, VM size, orion disk. Dixie gets
node_exporter, nvidia_gpu_exporter, and llama-server --metrics.
Tether labelled series still wait on titan's fix. Rule 2 is written
against reason=client_disconnect so it stays silent until then.
NM dispatcher plus a boot oneshot keep the AX9L (mt7921u) primary and
bring up the on-box iwlwifi profile if the dongle is gone. Connection
PSKs stay on-box; this only selects the radio.
Prefill ran in 2048-token scheduler steps of ~2.4 s; each decoding slot
gets one token per step, so guardian calls (16 tokens) waited behind
Honcho's 10-28k-token prompts: p90 10 s, max 14 s under two sift
digests (2026-09-23 05:50). ubatch is already 512, so prefill
throughput is unchanged; steps drop to ~0.6 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The unified 96k pool overflowed ~every 2-3 min under Honcho load
(39 events 2026-09-23 00:50-04:28). Each overflow fails every in-flight
slot with 'Context size has been exceeded', so Honcho retries and the
guardians' calls die with it. KV is ~20 KB/token on this model; 128k
costs ~0.65 GB of the ~2.5 GB free.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Honcho catch-up wave (48 deriver batches + 11 dreams + dialectic prefetches)
held all 6 slots with long thinking generations; tiny aux calls (title,
approval guardian) queued behind them and title_generation timed out at 60 s.
Two spare slots plus a bigger shared KV pool (so 6 long prompts can't exhaust
it). VRAM 9.7 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Room for the deriver, dialectic prefetches from three agents and their approval
guardians at once. Under full load (deriver + dialectic + guardian probes):
guardian p50 0.4 s / max 10 s, no KV-space failures. VRAM 8.9 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Dialectic benchmark (8 hermes-fleet questions): Qwen3-4B answered in 2-12 s but
often wrongly (invented facts, 'no information' on things memory holds); the 9B
answered correctly. All Honcho dialectic levels now use the 9B (rift .env). Its
VRAM goes to the 9B's KV pool: at 2 x 16k, concurrent deriver + dialectic hit
'failed to find free space in the KV cache'. 9B 6.3 GB + embed 2.2 GB = 8.4 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stock Ornith template raises 'No user query found in messages.' when a request
has no plain user turn, which Honcho's dialectic tool loop sends on later
iterations: 6 of 8 'low'-level dialectic calls failed with HTTP 500 when routed to
the 9B. The patched copy treats the last message as the query instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
kv-unified lets one request use a model's whole 16k (split slots capped each at
8k and every Honcho dialectic prompt of 11-17k tokens failed). 9B gets temp 0.6 /
top-p 0.95 / top-k 20 / min-p 0: in a 20-batch replay of Honcho's deriver the
0.8 default dropped 7 batches (empty or wrong-shape output); with these, 3,
and 1 with a format instruction on rift. VRAM unchanged at 11.5 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
8192/slot (previous commit) cost 6.6 GB of compute buffers and the 9B then
failed to load; 4096/slot did not fit beside the 9B and dialectic either.
2048/slot with batch-size set explicitly; long inputs are handled on rift
instead (EMBEDDING_MAX_INPUT_TOKENS=1536 makes Honcho chunk). All three
models resident at 11.5 GB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
batch-size defaulted to 2048 (capping ubatch) and ctx 8192 over 4 slots left
2048 per slot, so Honcho inputs over 2048 tokens failed with 'input is too
large to process'. Now batch/ubatch 8192, ctx 16384 over 2 slots = 8192/slot,
matching Honcho's EMBEDDING.MAX_INPUT_TOKENS. Verified a 6191-token input.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Alienware Aurora R11 (i7-10700, 64 GB, one RTX 3060 12 GB today; second card
when its fans are replaced). Headless, Wi-Fi only, unencrypted root by
decision (always-on box that must come back from a power cut unattended).
- llama.cpp router (CUDA, from unstable) on :11434 with the three helper
models resident (--models-max 3): ornith 9B for the Hermes approval guardian
/ title / web_extract and Honcho deriver, qwen3-4b for Honcho dialectic,
Qwen3-Embedding-0.6B for Honcho embeddings. Vision (qwen3-vl-8b) on its own
on-demand server on :11433 since it does not fit beside them in 12 GB.
- Both servers tailnet-only (tailscale0 firewall); GPUs capped at 140 W
(thermal hour 2026-09-21: 75 C plateau, no throttling).
- Model weights on the USB cache stick (ext4 label dixie-cache) bind-mounted
into /var/lib/llama-server/models; filled from straylight with dixie-push.
- 6.12 LTS kernel (mt7921u for the AX9L USB Wi-Fi arriving 2026-09-24).
- SSH keys baked in: no console after install to add them from.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
kyleandClaude Opus 5.5