VRAM guide

Best local LLMs for 12GB VRAM

The RTX 3060 12GB, RTX 4070, and RTX 5070 sit in the 12GB band. Ranked for GeForce RTX 3060 12GB at 12 GB. Updated 2026-10-24. These are estimates for llama.cpp with FP16 KV cache, not measured benchmarks. Method.

Model	Size	Verdict	Quant	tok/s
Qwen3 14B	14.8B	Tight fit	Q5_K_S	18–25
NousResearch Hermes 4 14B	14.8B	Tight fit	Q5_K_S	18–25
Ministral 3 14B Instruct 2512	13.5B	Tight fit	UD-Q5_K_XL	18–25
Ministral 3 14B Reasoning 2512	13.5B	Tight fit	UD-Q5_K_XL	18–25
Qwen2.5 14B Instruct	14.8B	Tight fit	Q4_K_M	19–25
DeepSeek R1 Distill Qwen 14B	14.8B	Tight fit	Q4_K_M	19–25
Qwen2.5 Coder 14B Instruct 128K	14.8B	Tight fit	Q4_K_M	19–25
phi 4	14.7B	Tight fit	Q4_K_M	19–26
gemma 4 12b it	11.9B	Tight fit	Q6_K	15–21
Qwen3.6 14B A3B FableVibes	13.8B	Tight fit	Q5_K_M	85–117
Devstral Small 2 24B Instruct 2512	23.6B	Tight fit	UD-IQ3_XXS	18–25
Mistral Small 3.2 24B Instruct 2506	23.6B	Tight fit	UD-IQ3_XXS	18–25
Magistral Small 2509	23.6B	Tight fit	UD-IQ3_XXS	18–25
Qwen3.8 27B	27.3B	Tight fit	UD-Q2_K_XL	19–26
Qwen2.5 32B Instruct	32.8B	Tight fit	IQ2_XXS	20–27
Qwen2.5 Coder 32B Instruct	32.8B	Tight fit	IQ2_XXS	20–27
Qwen3 32B	32.8B	Tight fit	UD-IQ2_XXS	19–26
Qwen3 VL 32B Instruct	32.8B	Tight fit	UD-IQ2_XXS	19–26
GLM 4 32B 0414	32.6B	Tight fit	UD-IQ2_XXS	20–28
GLM 4.6V Flash	9.4B	Tight fit	Q8_0	19–26
GLM Z1 9B 0414	9.4B	Tight fit	Q8_0	19–26
GLM 4 9B 0414	9.4B	Tight fit	Q8_0	19–26
gemma 4 31B it	30.7B	Tight fit	UD-IQ2_XXS	12–16
Qwen3.8 9B Distill	9.2B	Tight fit	Q8_0	20–27
Qwen3.5 9B	9.0B	Tight fit	Q8_0	20–28
MiMo V2.6 Distill Qwen 9B	9.0B	Tight fit	Q8_0	20–28
Qwythos 9B Claude Mythos 5 1M	9.0B	Tight fit	Q8_0	20–28
Qwythos 9B v2	9.0B	Tight fit	Q8_0	20–28
granite 4.2 8b	8.8B	Tight fit	Q8_0	19–25
Ministral 3 8B Instruct 2512	8.5B	Tight fit	Q8_0	19–27

102 models fit fully on this reference profile. 192-bit GDDR6 at 15 Gbps = 360 GB/s.