GGPUfit

unsloth

DeepSeek V3.2

DeepSeek V3.2 is a 671.0B mixture-of-experts deepseek2 model from unsloth, with about 121.6B parameters active per token. Configured context length is 163,840 tokens. License: mit. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.

Parameters671B / 122B active
Context160K
Attentionmla
Licensemit

61 layers · 128 KV heads · head dim 56 · GGUF repo · config

Quant file sizes

QuantFile sizeSource
BF161250.1 GBHugging Face file bytes
UD-Q8_K_XL726.7 GBHugging Face file bytes
Q8_0664.3 GBHugging Face file bytes
UD-Q6_K_XL534.6 GBHugging Face file bytes
Q6_K513.5 GBHugging Face file bytes
UD-Q5_K_XL448.8 GBHugging Face file bytes
Q5_K_M443.5 GBHugging Face file bytes
Q5_K_S430.9 GBHugging Face file bytes
UD-Q4_K_XL379.8 GBHugging Face file bytes
Q4_K_M377.6 GBHugging Face file bytes
Q4_K_S354.9 GBHugging Face file bytes
IQ4_NL353.1 GBHugging Face file bytes
IQ4_XS333.7 GBHugging Face file bytes
Q4_1391.9 GBHugging Face file bytes
Q4_0354.0 GBHugging Face file bytes
UD-Q3_K_XL299.0 GBHugging Face file bytes
Q3_K_M298.2 GBHugging Face file bytes
Q3_K_S270.5 GBHugging Face file bytes
UD-IQ3_XXS254.4 GBHugging Face file bytes
UD-Q2_K_XL229.7 GBHugging Face file bytes
Q2_K457.2 GBHugging Face file bytes
UD-IQ2_XXS202.2 GBHugging Face file bytes
UD-IQ1_M185.6 GBHugging Face file bytes
UD-IQ1_S171.5 GBHugging Face file bytes

On common machines

DeviceVerdictQuantSpeed
GeForce RTX 4060 Ti 8GBWon't fitUD-IQ1_S—
GeForce RTX 3060 12GBWon't fitUD-IQ1_S—
GeForce RTX 4060 Ti 16GBWon't fitUD-IQ1_S—
GeForce RTX 4090Won't fitUD-IQ1_S—
GeForce RTX 5090Won't fitUD-IQ1_S—
Apple M4 ProWon't fitUD-IQ1_S—
Apple M5Won't fitUD-IQ1_S—
CPU only · DDR5-5600Won't fitUD-IQ1_S—

Smallest devices that fit it fully

Run it

Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-IQ1_S).

llama.cpp
llama-server -hf unsloth/DeepSeek-V3.2-GGUF:UD-IQ1_S -c 2048 -ngl 0

No matching name in the Ollama library snapshot.

LM Studio
lms get unsloth/DeepSeek-V3.2-GGUF
# Then load quant UD-IQ1_S