straylight: llama-server router mode on llama.cpp 0.4.0

Drop the b9828 source pin and take llama.cpp from a refreshed
nixpkgs-unstable (0.4.0), which supports the Laguna and Gemma 4
architectures. Run llama-server in router mode with a preset file:
abliterated Qwen3.6-35B-A3B and Gemma 4 26B-A4B for benchmarking,
Poolside Laguna S 2.1 as the primary coding agent, and Ornith kept for
comparison. At most two models stay resident; idle models unload after
six hours. Add llama-models and llama-unload helper commands.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-09-15 19:54:51 -07:00
co-authored by Claude Fable 5.1
parent ec1e2efbba
commit 1714600aab
2 changed files with 84 additions and 28 deletions
Generated
+4 -4
View File
@@ -334,11 +334,11 @@
},
"nixpkgs-unstable": {
"locked": {
"lastModified": 1783224372,
"narHash": "sha256-FGQ8TfSm9WviNE/fQAWsKs+y4GkX8fU4EScewpw54Y8=",
"rev": "d407951447dcd00442e97087bf374aad70c04cea",
"lastModified": 1789286504,
"narHash": "sha256-3i80JnZ9pZ7tWReb/Sqpc32cfzffCu8TWLnbnH0hN9c=",
"rev": "ef34387ddd751e1ab8857adf4676492d32eb24ec",
"type": "tarball",
"url": "https://releases.nixos.org/nixos/unstable/nixos-26.11pre1027867.d407951447dc/nixexprs.tar.xz"
"url": "https://releases.nixos.org/nixos/unstable/nixos-26.11pre1073009.ef34387ddd75/nixexprs.tar.zst"
},
"original": {
"id": "nixpkgs",