GGPUfit

unsloth

Qwen3 235B A22B Instruct 2507

Qwen3 235B A22B Instruct 2507 is a 235.1B mixture-of-experts qwen3moe model from unsloth, with about 22.0B parameters active per token. Configured context length is 262,144 tokens. License: apache-2.0. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.

Parameters235B / 22.0B active
Context256K
Attentiongqa
Licenseapache-2.0

94 layers · 4 KV heads · head dim 128 · GGUF repo · config

Quant file sizes

QuantFile sizeSource
BF16438.0 GBHugging Face file bytes
UD-Q8_K_XL255.4 GBHugging Face file bytes
Q8_0232.8 GBHugging Face file bytes
UD-Q6_K_XL188.1 GBHugging Face file bytes
Q6_K179.8 GBHugging Face file bytes
UD-Q5_K_XL157.3 GBHugging Face file bytes
Q5_K_M155.4 GBHugging Face file bytes
Q5_K_S150.8 GBHugging Face file bytes
UD-Q4_K_XL125.1 GBHugging Face file bytes
Q4_K_M132.4 GBHugging Face file bytes
Q4_K_S124.5 GBHugging Face file bytes
IQ4_XS116.9 GBHugging Face file bytes
Q4_1137.1 GBHugging Face file bytes
Q4_0124.0 GBHugging Face file bytes
UD-Q3_K_XL97.0 GBHugging Face file bytes
Q3_K_M104.7 GBHugging Face file bytes
Q3_K_S94.5 GBHugging Face file bytes
UD-Q2_K_XL82.7 GBHugging Face file bytes
Q2_K159.7 GBHugging Face file bytes

On common machines

DeviceVerdictQuantSpeed
GeForce RTX 4060 Ti 8GBWon't fitUD-Q2_K_XL—
GeForce RTX 3060 12GBWon't fitUD-Q2_K_XL—
GeForce RTX 4060 Ti 16GBWon't fitUD-Q2_K_XL—
GeForce RTX 4090Won't fitUD-Q2_K_XL—
GeForce RTX 5090Won't fitUD-Q2_K_XL—
Apple M4 ProWon't fitUD-Q2_K_XL—
Apple M5Won't fitUD-Q2_K_XL—
CPU only · DDR5-5600Won't fitUD-Q2_K_XL—

Smallest devices that fit it fully

Run it

Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-Q2_K_XL).

llama.cpp
llama-server -hf unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF:UD-Q2_K_XL -c 2048 -ngl 0
Ollama
ollama run qwen3
LM Studio
lms get unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF
# Then load quant UD-Q2_K_XL