GGPUfit

unsloth

gemma 4 26B A4B it

gemma 4 26B A4B it is a 25.2B mixture-of-experts gemma4 model from unsloth, with about 4.0B parameters active per token. Configured context length is 262,144 tokens. License: apache-2.0. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.

Parameters25.2B / 4.0B active
Context256K
Attentionhybrid
Licenseapache-2.0

30 layers · 8 KV heads · head dim 256 · 25 sliding layers · GGUF repo · config

Quant file sizes

QuantFile sizeSource
BF1647.0 GBHugging Face file bytes
UD-Q8_K_XL25.7 GBHugging Face file bytes
Q8_025.0 GBHugging Face file bytes
UD-Q6_K_XL21.7 GBHugging Face file bytes
UD-Q6_K21.6 GBHugging Face file bytes
UD-Q5_K_XL19.8 GBHugging Face file bytes
UD-Q5_K_M19.7 GBHugging Face file bytes
UD-Q5_K_S17.6 GBHugging Face file bytes
UD-Q4_K_XL15.8 GBHugging Face file bytes
UD-Q4_K_M15.8 GBHugging Face file bytes
UD-Q4_K_S15.4 GBHugging Face file bytes
UD-IQ4_XS12.7 GBHugging Face file bytes
IQ4_NL12.7 GBHugging Face file bytes
UD-Q3_K_XL12.0 GBHugging Face file bytes
Q3_K_M11.9 GBHugging Face file bytes
UD-IQ3_S10.5 GBHugging Face file bytes
UD-IQ3_XXS10.6 GBHugging Face file bytes
UD-Q2_K_XL9.82 GBHugging Face file bytes
UD-IQ2_XXS9.24 GBHugging Face file bytes

On common machines

DeviceVerdictQuantSpeed
GeForce RTX 4060 Ti 8GBExpert offloadUD-Q4_K_M49–67
GeForce RTX 3060 12GBTight fitUD-Q2_K_XL74–101
GeForce RTX 4060 Ti 16GBTight fitUD-IQ4_XS38–52
GeForce RTX 4090Tight fitUD-Q5_K_XL103–141
GeForce RTX 5090Tight fitUD-Q8_K_XL154–210
Apple M4 ProTight fitUD-Q4_K_XL32–43
Apple M5Tight fitUD-Q2_K_XL23–31
CPU only · DDR5-5600Runs on CPUUD-Q6_K_XL8.6–12

Smallest devices that fit it fully

Run it

Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-IQ4_XS).

llama.cpp
llama-server -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-IQ4_XS -c 8192 -ngl 30
Ollama
ollama run gemma
LM Studio
lms get unsloth/gemma-4-26B-A4B-it-GGUF
# Then load quant UD-IQ4_XS