VRAM guide

Best local LLMs for 32GB

32GB discrete cards such as the RTX 5090, plus 32GB Macs. Ranked for GeForce RTX 5090 at 32 GB. Updated 2026-10-24. These are estimates for llama.cpp with FP16 KV cache, not measured benchmarks. Method.

Model	Size	Verdict	Quant	tok/s
Qwen2.5 32B Instruct	32.8B	Runs	Q5_K_M	39–53
Qwen2.5 Coder 32B Instruct	32.8B	Runs	Q5_K_M	39–53
DeepSeek R1 Distill Qwen 32B	32.8B	Tight fit	Q6_K	34–46
Qwen3 32B	32.8B	Tight fit	UD-Q6_K_XL	33–45
Qwen3 VL 32B Instruct	32.8B	Tight fit	UD-Q6_K_XL	33–45
GLM 4 32B 0414	32.6B	Tight fit	UD-Q6_K_XL	34–46
Qwen3 Coder Next	79.7B	Tight fit	UD-IQ3_S	169–230
DeepSeek R1 Distill Llama 70B	70.6B	Tight fit	UD-IQ3_XXS	32–44
Llama 3.3 70B Instruct	70.6B	Tight fit	UD-IQ3_XXS	32–44
Hermes 4 70B	70.6B	Tight fit	UD-IQ3_XXS	32–44
gemma 4 31B it	30.7B	Tight fit	UD-Q6_K_XL	28–38
Qwen3.8 27B	27.3B	Tight fit	UD-Q8_K_L	34–47
Swift Qwen3.8 27B	27.3B	Tight fit	Q8_0	33–45
gemma 3 27b it	27.0B	Tight fit	Q8_0	30–41
medgemma 27b text it	27.0B	Tight fit	Q8_0	30–41
Qwen3.6 27B	26.9B	Tight fit	Q8_0	34–46
Qwen3.5 27B	26.9B	Tight fit	Q8_0	34–46
Kwaipilot KAT Coder V2.5 Dev	34.7B	Runs	Q5_K_M	210–286
Cyber Tiel Coder 35B A3B GGUF MTP	35.5B	Tight fit	Q6_K	375–511
Qwen3.8 35B A3B Distill	35.5B	Tight fit	Q6_K	374–510
Tiel Coder 35B A3B	34.7B	Tight fit	UD-Q5_K_XL	399–544
Qwen3.5 35B A3B	34.7B	Tight fit	UD-Q6_K	374–510
Qwen3.6 35B A3B	34.7B	Tight fit	UD-Q6_K	364–497
NVIDIA Nemotron 3.5 Lightning 30B A3B	32.9B	Tight fit	UD-Q5_K_XL	307–419
Devstral Small 2 24B Instruct 2512	23.6B	Tight fit	UD-Q8_K_XL	32–44
Mistral Small 3.2 24B Instruct 2506	23.6B	Tight fit	UD-Q8_K_XL	32–44
Magistral Small 2509	23.6B	Tight fit	UD-Q8_K_XL	32–44
Nemotron 3 Nano 30B A3B	31.6B	Tight fit	UD-Q5_K_XL	323–441
NVIDIA Nemotron 3 Nano Omni 30B A3B Reasoning	31.6B	Tight fit	UD-Q5_K_XL	307–419
GLM 4.7 Flash	29.9B	Runs	UD-Q6_K_XL	164–224

117 models fit fully on this reference profile. NVIDIA official compare table: memory config and bandwidth.