GGPUfit

unsloth

MiMo V2 Flash

MiMo V2 Flash is a 308.8B mixture-of-experts mimo2 model from unsloth, with about 56.0B parameters active per token. Configured context length is 262,144 tokens. License: mit. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.

Parameters309B / 56.0B active
Context256K
Attentiongqa
Licensemit

48 layers · 4 KV heads · head dim 192 · GGUF repo · config

Quant file sizes

QuantFile sizeSource
BF16575.2 GBHugging Face file bytes
UD-Q8_K_XL329.9 GBHugging Face file bytes
Q8_0305.7 GBHugging Face file bytes
UD-Q6_K_XL245.3 GBHugging Face file bytes
Q6_K236.1 GBHugging Face file bytes
UD-Q5_K_XL204.7 GBHugging Face file bytes
Q5_K_M204.1 GBHugging Face file bytes
Q5_K_S197.9 GBHugging Face file bytes
UD-Q4_K_XL165.5 GBHugging Face file bytes
Q4_K_M174.0 GBHugging Face file bytes
Q4_K_S163.3 GBHugging Face file bytes
IQ4_NL162.1 GBHugging Face file bytes
IQ4_XS153.2 GBHugging Face file bytes
Q4_1180.0 GBHugging Face file bytes
Q4_0162.7 GBHugging Face file bytes
UD-Q3_K_XL127.7 GBHugging Face file bytes
Q3_K_M137.2 GBHugging Face file bytes
Q3_K_S124.0 GBHugging Face file bytes
UD-IQ3_XXS118.7 GBHugging Face file bytes
UD-Q2_K_XL108.3 GBHugging Face file bytes
Q2_K209.3 GBHugging Face file bytes
UD-IQ2_XXS94.6 GBHugging Face file bytes
UD-IQ1_M87.4 GBHugging Face file bytes
UD-IQ1_S81.2 GBHugging Face file bytes

On common machines

DeviceVerdictQuantSpeed
GeForce RTX 4060 Ti 8GBWon't fitUD-IQ1_S—
GeForce RTX 3060 12GBWon't fitUD-IQ1_S—
GeForce RTX 4060 Ti 16GBWon't fitUD-IQ1_S—
GeForce RTX 4090Won't fitUD-IQ1_S—
GeForce RTX 5090Won't fitUD-IQ1_S—
Apple M4 ProWon't fitUD-IQ1_S—
Apple M5Won't fitUD-IQ1_S—
CPU only · DDR5-5600Won't fitUD-IQ1_S—

Smallest devices that fit it fully

Run it

Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-IQ1_S).

llama.cpp
llama-server -hf unsloth/MiMo-V2-Flash-GGUF:UD-IQ1_S -c 2048 -ngl 0

No matching name in the Ollama library snapshot.

LM Studio
lms get unsloth/MiMo-V2-Flash-GGUF
# Then load quant UD-IQ1_S