GGPUfit

unsloth

GLM 4.7

GLM 4.7 is a 358.3B mixture-of-experts glm4moe model from unsloth, with about 71.7B parameters active per token. Configured context length is 202,752 tokens. License: mit. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.

Parameters358B / 71.7B active
Context198K
Attentiongqa
Licensemit

92 layers · 8 KV heads · head dim 128 · GGUF repo · config

Quant file sizes

QuantFile sizeSource
BF16667.6 GBHugging Face file bytes
UD-Q8_K_XL367.7 GBHugging Face file bytes
Q8_0354.8 GBHugging Face file bytes
UD-Q6_K_XL280.4 GBHugging Face file bytes
Q6_K274.2 GBHugging Face file bytes
UD-Q5_K_XL236.2 GBHugging Face file bytes
Q5_K_M236.8 GBHugging Face file bytes
Q5_K_S230.0 GBHugging Face file bytes
UD-Q4_K_XL190.5 GBHugging Face file bytes
Q4_K_M201.6 GBHugging Face file bytes
Q4_K_S189.7 GBHugging Face file bytes
IQ4_NL188.6 GBHugging Face file bytes
IQ4_XS178.4 GBHugging Face file bytes
Q4_1209.2 GBHugging Face file bytes
Q4_0189.1 GBHugging Face file bytes
UD-Q3_K_XL147.8 GBHugging Face file bytes
Q3_K_M159.5 GBHugging Face file bytes
Q3_K_S144.4 GBHugging Face file bytes
UD-IQ3_XXS135.2 GBHugging Face file bytes
UD-Q2_K_XL125.9 GBHugging Face file bytes
Q2_K244.5 GBHugging Face file bytes
UD-IQ2_XXS108.0 GBHugging Face file bytes
UD-IQ1_M100.3 GBHugging Face file bytes
UD-IQ1_S90.5 GBHugging Face file bytes

On common machines

DeviceVerdictQuantSpeed
GeForce RTX 4060 Ti 8GBWon't fitUD-IQ1_S—
GeForce RTX 3060 12GBWon't fitUD-IQ1_S—
GeForce RTX 4060 Ti 16GBWon't fitUD-IQ1_S—
GeForce RTX 4090Won't fitUD-IQ1_S—
GeForce RTX 5090Won't fitUD-IQ1_S—
Apple M4 ProWon't fitUD-IQ1_S—
Apple M5Won't fitUD-IQ1_S—
CPU only · DDR5-5600Won't fitUD-IQ1_S—

Smallest devices that fit it fully

Run it

Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-IQ1_S).

llama.cpp
llama-server -hf unsloth/GLM-4.7-GGUF:UD-IQ1_S -c 2048 -ngl 0

No matching name in the Ollama library snapshot.

LM Studio
lms get unsloth/GLM-4.7-GGUF
# Then load quant UD-IQ1_S