kyle/imladris
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
527 Commits 2 Branches 0 Tags
bccbe59e94817c3c79ae0e9d9a44055e081614e4
Commit Graph
527 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
kyle bccbe59e94 straylight: prefer IPv4 in gai.conf to fix update-channel timeouts 2026-10-11 12:05:40 -07:00
kyle fa35293331 sift: pin 682b9b8044 (restore force-pushed commit) 2026-10-09 18:43:36 -07:00
kyleandClaude Opus 5.5 6d1000c756 straylight: add libxi to nix-ld libraries 
Hermes' cua-driver (computer-use backend) is a prebuilt binary that links
libXi.so.6; without it the PM install fails verification with exit 127.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 18:43:18 -07:00
kyle b5e189bc5b llama-server: add Gemma4 31B and Muse-Glimmer-30B models 
Both fetched to /var/lib/llama-server/models by download.sh. Gemma4 31B
uses MTP draft (spec-type draft-mtp); Muse-Glimmer uses DFlash drafter
(spec-type draft-dflash, parallel=1 for the 104 GiB TTM window).
 2026-10-05 18:14:15 -07:00
kyle d7c1092d52 orion: pin sift 682b9b8 (P3.1 search filters + multi-query) 2026-09-30 07:38:31 -07:00
kyleandClaude Opus 5.5 c8b3b21004 dixie: honcho-embed pooling cls -> last 
Qwen3-Embedding is a causal model; its card pools the last token. With cls every
input embedded as its first token's state ("the cat sat on the mat" and "the stock
market crashed today" at cosine 1.0). A CPU side server with --pooling last
reproduced the card's example scores within 0.002 without a manual EOS. Honcho's
stored vectors and sift's embed cache are re-embedded / flushed after the switch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 23:46:37 -07:00
kyle 03ad1de40e straylight: add trinity-mini (Arcee Trinity Mini 26B MoE) to llama-server router 2026-09-27 00:56:41 -07:00
kyle cb0bc87633 orion: pin sift 5a59f7a (P6 asynchronous batch fetch) 2026-09-26 23:32:13 -07:00
kyle 4fa779be9c orion: pin sift bdc093e (P5b long-doc questions + P5c extract; P6 plan docs) 2026-09-26 20:22:32 -07:00
kyle cd2bd3d0e3 orion: pin sift 1311cf3 (P5a PDF documents) 
PDF extraction in a re-exec'd child, per-page cache, map/pages/
grep, cited digests. Module MemoryMax 512M -> 1G via pinned sift.
 2026-09-26 10:57:22 -07:00
kyle ec39a569de dixie: Z tmpfiles rules use ~0755 so existing files stay 0644 (exec dropped), dirs become 0755 2026-09-25 07:49:51 -07:00
kyle e5c47007cd dixie: Z tmpfiles rules to relabel dropsonde2 trees 0755 recursively (fixes rsync's 0700 dirs) 2026-09-25 07:48:32 -07:00
kyle 9a5920da2b dixie: drop uid/gid from rsyncd modules — daemon runs as dropsonde via SSH, setgroups() fails otherwise 2026-09-25 03:03:40 -07:00
kyle 2305f4099a dixie: rsync --server --daemon sync key (no rrsync/python3 on dixie) 
- authorized key forces rsync --server --daemon --config=/etc/rsyncd-dropsonde.conf
- environment.etc rsyncd-dropsonde.conf: cold + analysis modules, read only=false,
  use chroot=false, uid/gid dropsonde, max connections 2, lock file global
- tmpfiles: /srv/{cold,analysis}/dropsonde2 owned dropsonde
 2026-09-25 02:45:44 -07:00
kyle 2e4aa6ed57 dixie: dropsonde sync key — rrsync -wo /srv (write-only, hyperborea -> dixie) 2026-09-25 02:24:26 -07:00
kyle 0ac90515bf dixie: use script attr for dropsonde-pg-setup (ExecStart heredoc+bash path broken) 2026-09-25 00:51:32 -07:00
kyle c402c1c599 dixie: drop python3.withPackages (duckdb CLI only) to trim the closure 2026-09-25 00:27:38 -07:00
kyle eed2821cea dixie: slice submodule uses sliceConfig, not serviceConfig 2026-09-24 23:42:12 -07:00
kyle 43b8e2288f dixie: listen_addresses under settings, not top-level 2026-09-24 23:41:40 -07:00
kyle 3980242da5 dixie: close the ExecStart indented string in dropsonde-pg-setup 2026-09-24 23:41:25 -07:00
kyle 4b751796c6 dixie: import analysis.nix (was never wired into the module list) 2026-09-24 23:36:31 -07:00
kyle 0c37b07099 dixie: label /srv/cold as ds2-cold, use sfdisk for wipe 2026-09-24 23:20:01 -07:00
kyle fddaffc825 dixie: fix slice unit name, pg-setup ExecStart, listen_addresses 2026-09-24 22:27:12 -07:00
kyle a40873685a dixie: open Postgres 5432 on tailscale0 for dropsonde 2026-09-24 22:17:44 -07:00
kyle 3c2a272722 dixie: dropsonde2 analysis tier (HDD /srv/cold, pg17+timescale+postgis+h3, analysis.slice, duckdb/polars) 2026-09-24 22:03:44 -07:00
kyleandClaude Opus 5.5 c12b2adbfd disk-health plan: deploy via committed master + doas (per CLAUDE.md) 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 18:59:39 -07:00
kyleandClaude Opus 5.5 85d5631fa8 docs: disk-health monitoring plan (SMART/NVMe/SD card) 
Current state verified on 2026-09-24: hyperborea NVMe already exported
via the Debian nvme textfile; no SMART on the NixOS hosts; no
disk-health alert rules. Plan: smartctl_exporter module for the NixOS
hosts, smartmon timer and SD-card kernel-error textfile on hyperborea,
orion scrape job, a disk-health rule group, verification and deploy
notes. For hand-off to another agent.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-09-24 18:59:15 -07:00
kyle 78d30026ce orion: allowlist vmui's /prometheus API prefix 
Default Server URL is /prometheus; keep import/admin 403.
 2026-09-24 01:56:05 -07:00
kyle f5ef78baca orion: read-only vmui via nginx allowlist + serve :8429 
VictoriaMetrics stays loopback :8428. nginx :8429 allows GET/POST
to /vmui and query APIs only; import/admin/export/flags 403.
tailscale serve HTTPS 8429. Review, no switch.
 2026-09-24 01:47:25 -07:00
kyle 1cbde84601 rift: linger kyle and enable user podman-restart 
Keep rootless podman up after logind idle.
 2026-09-24 01:33:19 -07:00
kyle f32db3c88b metrics: systemd unit-include as [.]service 
systemd drops \. in unit-include and warns about unknown escapes.
 2026-09-24 01:22:16 -07:00
kyle 81897b3f8a metrics: keep llama-straylight out of FleetTargetDown 
35B eviction is not a paging event. params.model stays.
 2026-09-24 00:46:23 -07:00
kyle 385529da39 metrics: phase 3 straylight/rift scrapes and hyperborea units 
Review, no switch. node-laptop for talos/titan is excluded from
FleetTargetDown. llama-straylight scrape uses params.model for
ornith-1.5-35b-a3b (router 400 without it; will load if cold).
 2026-09-24 00:42:06 -07:00
kyle b861c6ee9f metrics: drop llama-dixie-vision scrape 
Router /metrics cannot be up without loading a model.
 2026-09-24 00:29:24 -07:00
kyle ed5e1a1e0e metrics: HonchoDeriverBacklog and include honcho in target-down 2026-09-24 00:21:36 -07:00
kyle 5d8e8c4335 metrics: scrape dixie helper /metrics with model param 
Router-mode llama-server returns 400 without it.
 2026-09-23 23:57:36 -07:00
kyle b1d5aa7e64 metrics: drop alertmanager channel override 
Webhook is already bound to ~alerts; Mattermost's channel
override wants the name, not ~alerts.
 2026-09-23 23:44:24 -07:00
kyle ec74e0b69c metrics: phase 1 follow-up (review, no switch) 
Honcho workspace_name keep, node_exporter restart metrics,
drop min_over_time on rule 1, Watchdog heartbeat, pin tether
to b267e2f so labelled series exist and rule 2 can fire.
 2026-09-23 23:43:11 -07:00
kyle 4958493485 metrics: phase 1 orion+dixie (review, no switch) 
VictoriaMetrics on 127.0.0.1:8428. LoadCredential for the Mattermost
webhook and the sift scrape token. Honcho scrapes keep a closed metric
name set. Meta-alerts for target-down, VM size, orion disk. Dixie gets
node_exporter, nvidia_gpu_exporter, and llama-server --metrics.

Tether labelled series still wait on titan's fix. Rule 2 is written
against reason=client_disconnect so it stays silent until then.
 2026-09-23 23:22:38 -07:00
kyle 888e8e24fb orion: pin sift 4372673 (P2.7 metrics) 
Prometheus /metrics behind a dedicated metrics caller; JSON stays
at ?format=json. No search-source change.
 2026-09-23 23:11:30 -07:00
kyle c07c4bfe4f docs: fix tether cancel-storm PromQL (rule 2) 
Label-set mismatch made the ratio empty. Select reason=client_disconnect
and kind=converted; never sum() a tether series without a selector.
 2026-09-23 22:50:26 -07:00
kyle 8b14e1f99f docs: fleet metrics and alerting plan 
Reviewed plan only: VictoriaMetrics on orion, vmalert/Alertmanager,
tailnet exporters, first five rules. No config, no deploy.
 2026-09-23 22:48:51 -07:00
kyle 504d4eb3f1 orion: pin sift e62cfc8 (restricted tavily cap skip) 2026-09-23 16:38:41 -07:00
kyle bc30315344 orion: sift tavily+searxng, cap 130, tavily no_rerank 
Kyle chose Tavily-first with SearXNG fallback. Brave stays off.
 2026-09-23 16:25:01 -07:00
kyle 75b813b10b orion: SearXNG-only until Kyle picks sift search default 2026-09-23 16:20:38 -07:00
kyle 415589cc13 orion: pin sift 1c57845, brave_llm only for live fixture 
RRF original-URL fix. SearXNG and Tavily stay off until Kyle picks a default.
 2026-09-23 16:10:59 -07:00
kyle 573b9dcfd8 orion: disable sift brave_llm for bake-off config 1 2026-09-23 15:42:54 -07:00
kyle c84ca93024 orion: disable sift tavily for bake-off config 2 2026-09-23 15:36:33 -07:00
kyle 91c021cff9 orion: enable sift searxng, brave_llm, tavily 
Bake-off sources. SearXNG journal is already LogLevelMax=err.
Keys stay in /var/lib/sift/secret.env, not the store.
 2026-09-23 14:14:25 -07:00
kyle c839f7c8ae dixie: fail over USB Wi-Fi to internal CNVi 
NM dispatcher plus a boot oneshot keep the AX9L (mt7921u) primary and
bring up the on-box iwlwifi profile if the dongle is gone. Connection
PSKs stay on-box; this only selects the radio.
 2026-09-23 14:03:30 -07:00