metrics: phase 1 orion+dixie (review, no switch)

VictoriaMetrics on 127.0.0.1:8428. LoadCredential for the Mattermost
webhook and the sift scrape token. Honcho scrapes keep a closed metric
name set. Meta-alerts for target-down, VM size, orion disk. Dixie gets
node_exporter, nvidia_gpu_exporter, and llama-server --metrics.

Tether labelled series still wait on titan's fix. Rule 2 is written
against reason=client_disconnect so it stays silent until then.
This commit is contained in:
2026-09-23 23:22:38 -07:00
parent 888e8e24fb
commit 4958493485
4 changed files with 236 additions and 1 deletions
+10 -1
View File
@@ -116,6 +116,7 @@ let
--port ${toString port} \
--models-preset ${preset} \
--models-max ${toString modelsMax} \
--metrics \
--timeout 7200
'';
User = "llama-server";
@@ -233,6 +234,7 @@ in
./hardware-configuration.nix
./disk-config.nix
inputs.tether.nixosModules.tether
../../configs/metrics/exporters.nix
];
config = {
@@ -323,7 +325,14 @@ in
# Reachable only over the tailnet: 11434 (router), 11433 (vision), 11435 (tether)
# on tailscale0, and SSH from the LAN (base config opens 22) for bring-up.
networking.firewall.interfaces.tailscale0.allowedTCPPorts = [ 11433 11434 11435 ];
networking.firewall.interfaces.tailscale0.allowedTCPPorts = [ 11433 11434 11435 9835 ];
services.prometheus.exporters.nvidia-gpu = {
enable = true;
listenAddress = "0.0.0.0";
port = 9835;
openFirewall = false;
};
environment.systemPackages = with pkgs; [
llama-cpp