GGPUfit

unsloth

Qwen3.6 35B A3B

Qwen3.6 35B A3B is a 34.7B mixture-of-experts qwen35moe model from unsloth, with about 3.0B parameters active per token. Configured context length is 262,144 tokens. License: apache-2.0. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.

Parameters34.7B / 3.0B active
Context256K
Attentionhybrid
Licenseapache-2.0

40 layers · 2 KV heads · head dim 256 · 30 linear-attention layers · GGUF repo · config

Quant file sizes

QuantFile sizeSource
BF1664.6 GBHugging Face file bytes
UD-Q8_K_XL35.8 GBHugging Face file bytes
Q8_034.4 GBHugging Face file bytes
UD-Q6_K_XL29.7 GBHugging Face file bytes
UD-Q6_K27.3 GBHugging Face file bytes
UD-Q5_K_XL24.8 GBHugging Face file bytes
UD-Q5_K_M24.6 GBHugging Face file bytes
UD-Q5_K_S23.2 GBHugging Face file bytes
UD-Q4_K_XL20.8 GBHugging Face file bytes
UD-Q4_K_M20.6 GBHugging Face file bytes
UD-Q4_K_S19.5 GBHugging Face file bytes
UD-IQ4_XS16.5 GBHugging Face file bytes
IQ4_NL35.0 GBHugging Face file bytes
UD-Q3_K_XL15.7 GBHugging Face file bytes
Q3_K_M15.5 GBHugging Face file bytes
UD-IQ3_S12.7 GBHugging Face file bytes
Q3_K_S14.3 GBHugging Face file bytes
UD-IQ3_XXS12.3 GBHugging Face file bytes
UD-Q2_K_XL11.4 GBHugging Face file bytes
UD-IQ2_XXS10.0 GBHugging Face file bytes
UD-IQ1_M9.36 GBHugging Face file bytes

On common machines

DeviceVerdictQuantSpeed
GeForce RTX 4060 Ti 8GBExpert offloadUD-Q4_K_M75–102
GeForce RTX 3060 12GBTight fitUD-IQ2_XXS180–246
GeForce RTX 4060 Ti 16GBTight fitUD-IQ3_S117–160
GeForce RTX 4090Tight fitUD-Q4_K_XL264–359
GeForce RTX 5090Tight fitUD-Q6_K364–497
Apple M4 ProTight fitUD-IQ4_XS88–120
Apple M5Tight fitUD-IQ2_XXS77–104
CPU only · DDR5-5600Runs on CPUUD-Q5_K_S21–29

Smallest devices that fit it fully

Run it

Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-IQ3_S).

llama.cpp
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-IQ3_S -c 8192 -ngl 40
Ollama
ollama run qwen3.6
LM Studio
lms get unsloth/Qwen3.6-35B-A3B-GGUF
# Then load quant UD-IQ3_S