kyle/imladris
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

straylight: raise the GPU memory ceiling to 104 GiB

Browse Source 
The TTM default caps GTT at half of RAM (~62 GiB), which is too small for
the 118B-parameter model straylight will serve. Set ttm.pages_limit and
ttm.page_pool_size to 27262976 pages (104 GiB); takes effect on reboot.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-15 19:54:51 -07:00
co-authored by Claude Fable 5.1
parent 7ba99f1dd7
commit ec1e2efbba
1 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hw/straylight/default.nix
+5
View File
@@ -25,6 +25,11 @@ in
]; ];
config = { config = {
# Let the GPU address up to 104 GiB of the 128 GiB of unified memory (the TTM default
# caps GTT at 50% of RAM, ~62 GiB). Units are 4 KiB pages; takes effect after a reboot.
# amdgpu.gttsize is deprecated on recent kernels; ttm.* is the supported knob.
boot.kernelParams = [ "ttm.pages_limit=27262976" "ttm.page_pool_size=27262976" ];
# llama.cpp server with Vulkan backend for gfx1151 (Strix Halo). # llama.cpp server with Vulkan backend for gfx1151 (Strix Halo).
# Model is downloaded from HuggingFace on first start and cached in StateDirectory. # Model is downloaded from HuggingFace on first start and cached in StateDirectory.
# NOTE: BIOS "UMA Frame Buffer Size" must be ≥32GB for a 23GB model to fit on-GPU. # NOTE: BIOS "UMA Frame Buffer Size" must be ≥32GB for a 23GB model to fit on-GPU.