GGPUfit

unsloth

Qwen3.5 35B A3B

Qwen3.5 35B A3B is a 34.7B mixture-of-experts qwen35moe model from unsloth, with about 3.0B parameters active per token. Configured context length is 262,144 tokens. License: apache-2.0. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.

Parameters34.7B / 3.0B active
Context256K
Attentionhybrid
Licenseapache-2.0

40 layers · 2 KV heads · head dim 256 · 30 linear-attention layers · GGUF repo · config

Quant file sizes

QuantFile sizeSource
BF1664.6 GBHugging Face file bytes
UD-Q8_K_XL45.3 GBHugging Face file bytes
Q8_034.4 GBHugging Face file bytes
UD-Q6_K_XL29.9 GBHugging Face file bytes
UD-Q6_K26.6 GBHugging Face file bytes
Q6_K26.9 GBHugging Face file bytes
UD-Q5_K_XL24.6 GBHugging Face file bytes
Q5_K_M24.4 GBHugging Face file bytes
Q5_K_S23.1 GBHugging Face file bytes
UD-Q4_K_XL20.7 GBHugging Face file bytes
Q4_K_M20.5 GBHugging Face file bytes
Q4_K_S19.3 GBHugging Face file bytes
UD-IQ4_XS16.3 GBHugging Face file bytes
IQ4_NL16.6 GBHugging Face file bytes
UD-Q3_K_XL15.5 GBHugging Face file bytes
Q3_K_M15.2 GBHugging Face file bytes
UD-IQ3_S12.7 GBHugging Face file bytes
Q3_K_S14.2 GBHugging Face file bytes
UD-IQ3_XXS12.2 GBHugging Face file bytes
UD-Q2_K_XL11.3 GBHugging Face file bytes
UD-IQ2_XXS9.93 GBHugging Face file bytes

On common machines

DeviceVerdictQuantSpeed
GeForce RTX 4060 Ti 8GBExpert offloadQ4_K_M75–103
GeForce RTX 3060 12GBTight fitUD-IQ2_XXS182–248
GeForce RTX 4060 Ti 16GBTight fitUD-IQ3_S118–161
GeForce RTX 4090Tight fitUD-Q4_K_XL265–361
GeForce RTX 5090Tight fitUD-Q6_K374–510
Apple M4 ProTight fitUD-IQ4_XS89–122
Apple M5Tight fitUD-IQ2_XXS77–105
CPU only · DDR5-5600Runs on CPUQ5_K_S21–29

Smallest devices that fit it fully

Run it

Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-IQ3_S).

llama.cpp
llama-server -hf unsloth/Qwen3.5-35B-A3B-GGUF:UD-IQ3_S -c 8192 -ngl 40
Ollama
ollama run qwen3.5
LM Studio
lms get unsloth/Qwen3.5-35B-A3B-GGUF
# Then load quant UD-IQ3_S