unsloth

LFM2.5 24B A2B

LFM2.5 24B A2B is a 24.0B mixture-of-experts lfm2moe model from unsloth, with about 2.0B parameters active per token. Configured context length is 131,072 tokens. License: other. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.

Parameters24.0B / 2.0B active

Context128K

Attentiongqa

Licenseother

40 layers · 8 KV heads · head dim 64 · GGUF repo · config

Quant file sizes

Quant	File size	Source
BF16	44.7 GB	Hugging Face file bytes
Q8_0	23.7 GB	Hugging Face file bytes
UD-Q6_K_XL	20.4 GB	Hugging Face file bytes
UD-Q5_K_XL	16.7 GB	Hugging Face file bytes
UD-Q4_K_XL	13.9 GB	Hugging Face file bytes
UD-Q3_K_XL	10.2 GB	Hugging Face file bytes
UD-IQ3_XXS	8.34 GB	Hugging Face file bytes
UD-Q2_K_XL	7.49 GB	Hugging Face file bytes
UD-IQ2_XXS	6.96 GB	Hugging Face file bytes

On common machines

Device	Verdict	Quant	Speed
GeForce RTX 4060 Ti 8GB	Expert offload	UD-Q4_K_XL	83–113
GeForce RTX 3060 12GB	Tight fit	UD-IQ3_XXS	140–190
GeForce RTX 4060 Ti 16GB	Runs	UD-Q3_K_XL	100–136
GeForce RTX 4090	Tight fit	UD-Q6_K_XL	222–303
GeForce RTX 5090	Runs	Q8_0	352–480
Apple M4 Pro	Tight fit	UD-Q5_K_XL	82–112
Apple M5	Tight fit	UD-IQ3_XXS	59–81
CPU only · DDR5-5600	Runs on CPU	Q8_0	20–27

Smallest devices that fit it fully

GeForce RTX 3080 10GB — Tight fit, UD-Q2_K_XL, 311–425 tok/s
Intel Arc B570 — Tight fit, UD-Q2_K_XL, 156–212 tok/s
GeForce RTX 2080 Ti — Tight fit, UD-IQ3_XXS, 239–326 tok/s
GeForce GTX 1080 Ti — Tight fit, UD-IQ3_XXS, 188–256 tok/s
GeForce RTX 5070 — Tight fit, UD-IQ3_XXS, 261–355 tok/s
GeForce RTX 4070 Ti — Tight fit, UD-IQ3_XXS, 195–266 tok/s

Run it

Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-Q3_K_XL).

llama.cpp

llama-server -hf unsloth/LFM2.5-24B-A2B-GGUF:UD-Q3_K_XL -c 8192 -ngl 40

Ollama

ollama run lfm2.5

LM Studio

lms get unsloth/LFM2.5-24B-A2B-GGUF
# Then load quant UD-Q3_K_XL