GGPUfit

VRAM guide

Best local LLMs for 8GB VRAM

Cards in this class include the RTX 4060 Ti 8GB, RTX 4060, RTX 3070, and Arc A750. Ranked for GeForce RTX 4060 Ti 8GB at 8 GB. Updated 2026-10-24. These are estimates for llama.cpp with FP16 KV cache, not measured benchmarks. Method.

ModelSizeVerdictQuanttok/s
GLM 4.6V Flash9.4BTight fitQ5_K_S23–31
GLM Z1 9B 04149.4BTight fitQ5_K_S23–31
GLM 4 9B 04149.4BTight fitQ5_K_S23–31
Qwen3.8 9B Distill9.2BTight fitQ5_K_M23–31
Qwythos 9B Claude Mythos 5 1M9.0BTight fitQ5_K_M24–32
Qwythos 9B v29.0BTight fitQ5_K_M23–32
Qwen3.5 9B9.0BTight fitUD-Q5_K_XL23–31
MiMo V2.6 Distill Qwen 9B9.0BTight fitQ5_K_M23–31
granite 4.2 8b8.8BTight fitQ5_K_M23–31
gemma 4 12b it11.9BTight fitIQ4_XS20–27
Ministral 3 8B Instruct 25128.5BTight fitUD-Q5_K_XL24–33
Ministral 3 8B Reasoning 25128.5BTight fitUD-Q5_K_XL24–33
Qwen3.8 27B27.3BTight fitUD-IQ1_M23–31
Qwen3 VL 8B Instruct8.2BTight fitUD-Q5_K_XL22–30
DeepSeek R1 0528 Qwen3 8B8.2BTight fitUD-Q5_K_XL22–30
Qwen3 8B8.2BTight fitUD-Q5_K_XL22–30
gemma 3 27b it27.0BTight fitUD-IQ1_S23–31
Qwen3 VL 8B Thinking8.2BTight fitUD-Q5_K_XL22–31
medgemma 27b text it27.0BTight fitUD-IQ1_S21–29
DeepSeek R1 Distill Llama 8B8.0BTight fitUD-Q5_K_XL23–32
Llama 3.1 8B Instruct8.0BTight fitUD-Q5_K_XL23–32
Apertus 8B Instruct 25098.1BTight fitUD-Q5_K_XL23–31
Qwen2.5 7B Instruct7.6BRunsQ5_K_M27–37
Qwen2.5 VL 7B Instruct7.6BTight fitQ6_K24–32
DeepSeek R1 Distill Qwen 7B7.6BTight fitQ6_K24–32
Qwen2.5 Coder 7B Instruct 128K7.6BTight fitQ6_K24–32
NVIDIA Nemotron 3.5 Bolt E4B7.5BRunsUD-Q5_K_XL24–33
gemma 4 E4B it7.5BTight fitUD-Q5_K_XL22–29
Devstral Small 2 24B Instruct 251223.6BTight fitUD-IQ1_M24–32
Mistral Small 3.2 24B Instruct 250623.6BTight fitUD-IQ1_M24–32

79 models fit fully on this reference profile. 8GB variant. Same 288 GB/s bus as the 16GB card.