kyle/imladris
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

straylight: add Qwen3-VL-8B abliterated as the small VLM

Browse Source 
Ornith 1.5 9B is a dense text model, not vision. Replace that role with
prithivMLmods Qwen3-VL-8B Instruct abliterated v2 (mradermacher Q4_K_M)
and pin mmproj-url so router mode actually loads the projector.
This commit is contained in:
kyle
2026-09-16 06:03:51 -07:00
parent 85d216fce3
commit 56b233afb6
1 changed files with 10 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hw/straylight/default.nix
+10 -2
View File
@@ -50,12 +50,20 @@ let
hf-repo = deepreinforce-ai/Ornith-1.0-35B-GGUF:Q4_K_M
dedup-cache-models = true
; Ornith 1.5 9B, refusal-ablated (junafinity / mradermacher GGUF). Multimodal:
; llama-server pulls mmproj with --hf-repo. Q4_K_M is the recommended quant.
; Ornith 1.5 9B, refusal-ablated (junafinity / mradermacher GGUF). Dense
; text/coding model — not a VLM; the shipped mmproj does not add vision.
[ornith-1.5-9b-uncensored]
hf-repo = mradermacher/Ornith-1.5-9B-uncensored-GGUF:Q4_K_M
dedup-cache-models = true
; Qwen3-VL-8B Instruct, refusal-ablated (prithivMLmods v2 / mradermacher).
; Dedicated VLM in the same size class as Ornith 1.5 9B. Explicit mmproj-url
; because router-mode --hf-repo does not attach the projector on its own.
[qwen3-vl-8b-abliterated]
hf-repo = mradermacher/Qwen3-VL-8B-Instruct-abliterated-v2.0-GGUF:Q4_K_M
mmproj-url = https://huggingface.co/mradermacher/Qwen3-VL-8B-Instruct-abliterated-v2.0-GGUF/resolve/main/Qwen3-VL-8B-Instruct-abliterated-v2.0.mmproj-f16.gguf
dedup-cache-models = true
; Ornith 1.5 35B-A3B MoE (official GGUF). ~3B active, multimodal; mmproj
; comes with --hf-repo. Q4_K_M is ~22 GB.
[ornith-1.5-35b-a3b]