kyle/imladris
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

straylight: add Ornith 1.5 35B-A3B MoE to llama-server

Browse Source 
Official ornith-ai Q4_K_M GGUF (~22 GB). Multimodal 35B MoE with ~3B
active; llama-server fetches weights and mmproj on first load.
This commit is contained in:
kyle
2026-09-16 05:56:27 -07:00
parent 5a734118b0
commit 85d216fce3
1 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hw/straylight/default.nix
+6
View File
@@ -56,6 +56,12 @@ let
hf-repo = mradermacher/Ornith-1.5-9B-uncensored-GGUF:Q4_K_M hf-repo = mradermacher/Ornith-1.5-9B-uncensored-GGUF:Q4_K_M
dedup-cache-models = true dedup-cache-models = true
; Ornith 1.5 35B-A3B MoE (official GGUF). ~3B active, multimodal; mmproj
; comes with --hf-repo. Q4_K_M is ~22 GB.
[ornith-1.5-35b-a3b]
hf-repo = ornith-ai/Ornith-1.5-35B-A3B-GGUF:Q4_K_M
dedup-cache-models = true
; Qwen 3.8 27B dense, refusal-ablated. Preferred uncensored coding agent. ; Qwen 3.8 27B dense, refusal-ablated. Preferred uncensored coding agent.
; orcarouter/Qwen3.8-27B-Uncensored-GGUF is gated; huihui Q4_K_L keeps ; orcarouter/Qwen3.8-27B-Uncensored-GGUF is gated; huihui Q4_K_L keeps
; ablation-target tensors at Q8_0. llama-server fetches on first load. ; ablation-target tensors at Q8_0. llama-server fetches on first load.