VRAM guide

Best local LLMs for 24GB VRAM

24GB is the RTX 3090, RTX 4090, and RX 7900 XTX class. Ranked for GeForce RTX 4090 at 24 GB. Updated 2026-10-24. These are estimates for llama.cpp with FP16 KV cache, not measured benchmarks. Method.

Model	Size	Verdict	Quant	tok/s
GLM 4 32B 0414	32.6B	Tight fit	Q5_K_S	24–33
gemma 4 31B it	30.7B	Tight fit	Q5_K_S	22–30
Qwen3 32B	32.8B	Tight fit	UD-Q4_K_XL	25–34
Qwen3 VL 32B Instruct	32.8B	Tight fit	UD-Q4_K_XL	25–34
Qwen2.5 32B Instruct	32.8B	Tight fit	Q4_K_M	25–34
Qwen2.5 Coder 32B Instruct	32.8B	Tight fit	Q4_K_M	25–34
DeepSeek R1 Distill Qwen 32B	32.8B	Tight fit	Q4_K_M	25–34
Swift Qwen3.8 27B	27.3B	Tight fit	Q5_K_M	27–36
Qwen3.8 27B	27.3B	Tight fit	UD-Q6_K	25–34
gemma 3 27b it	27.0B	Tight fit	Q6_K	23–31
medgemma 27b text it	27.0B	Tight fit	Q6_K	23–31
Qwen3.6 27B	26.9B	Tight fit	Q6_K	24–33
Qwen3.5 27B	26.9B	Tight fit	Q6_K	24–33
Devstral Small 2 24B Instruct 2512	23.6B	Tight fit	UD-Q6_K_XL	25–34
Mistral Small 3.2 24B Instruct 2506	23.6B	Tight fit	UD-Q6_K_XL	25–34
Magistral Small 2509	23.6B	Tight fit	UD-Q6_K_XL	25–34
Qwen3.5 35B A3B	34.7B	Tight fit	UD-Q4_K_XL	265–361
Qwen3.6 35B A3B	34.7B	Tight fit	UD-Q4_K_XL	264–359
Tiel Coder 35B A3B	34.7B	Tight fit	UD-Q4_K_XL	264–359
GLM 4.7 Flash	29.9B	Tight fit	UD-Q5_K_XL	110–150
Kwaipilot KAT Coder V2.5 Dev	34.7B	Tight fit	Q4_K_M	137–187
Cyber Tiel Coder 35B A3B GGUF MTP	35.5B	Tight fit	Q4_K_M	279–381
Qwen3.8 35B A3B Distill	35.5B	Tight fit	Q4_K_M	277–378
Qwen3 Coder 30B A3B Instruct	30.5B	Tight fit	UD-Q5_K_XL	188–257
Qwen3 30B A3B Instruct 2507	30.5B	Tight fit	UD-Q5_K_XL	188–257
Qwen3 VL 30B A3B Instruct	30.5B	Tight fit	UD-Q5_K_XL	188–257
Qwen3 30B A3B Thinking 2507	30.5B	Tight fit	UD-Q5_K_XL	188–257
DeepSeek R1 Distill Llama 70B	70.6B	Tight fit	UD-IQ2_XXS	25–34
Llama 3.3 70B Instruct	70.6B	Tight fit	UD-IQ2_XXS	25–34
Hermes 4 70B	70.6B	Tight fit	UD-IQ2_XXS	25–34

115 models fit fully on this reference profile. NVIDIA GeForce RTX 40-series specifications. Bandwidth is the published peak.