Hardware

NVIDIA A10

NVIDIA A10 has 24 GB GDDR6 and 600 GB/s. See which open models fit, at which quant, and roughly how fast. Figures below use the default memory (24 GB), 32 GB system RAM, and DDR5-5600 for anything that spills. Updated 2026-10-17.

Memory24 GB GDDR6

Bandwidth600 GB/s

Kindgpu

Year2021

24GB GDDR6, 600 GB/s. Source.

RSS for new models on this device

Models that fit

Model	Verdict	Quant	Context	tok/s
GLM 4 32B 0414 32.6B	Tight fit	UD-Q5_K_XL	4K	14–19
gemma 4 31B it 30.7B	Tight fit	UD-Q5_K_XL	4K	13–17
Qwen3 32B 32.8B	Tight fit	UD-Q4_K_XL	8K	15–20
Qwen3 VL 32B Instruct 32.8B	Tight fit	UD-Q4_K_XL	8K	15–20
Qwen2.5 32B Instruct 32.8B	Tight fit	Q4_K_M	8K	15–20
Qwen2.5 Coder 32B Instruct 32.8B	Tight fit	Q4_K_M	8K	15–20
DeepSeek R1 Distill Qwen 32B 32.8B	Tight fit	Q4_K_M	8K	15–20
Qwen3.8 27B 27.3B	Tight fit	UD-Q6_K_M	4K	14–19
gemma 3 27b it 27.0B	Tight fit	Q6_K	8K	13–17
medgemma 27b text it 27.0B	Tight fit	Q6_K	8K	13–17
Qwen3.6 27B 26.9B	Tight fit	Q6_K	8K	14–20
Qwen3.5 27B 26.9B	Tight fit	Q6_K	8K	14–20
Devstral Small 2 24B Instruct 2512 23.6B	Tight fit	UD-Q6_K_XL	8K	15–20
Mistral Small 3.2 24B Instruct 2506 23.6B	Tight fit	UD-Q6_K_XL	8K	15–20
Magistral Small 2509 23.6B	Tight fit	UD-Q6_K_XL	8K	15–20
Qwen3.5 35B A3B 34.7B	Tight fit	UD-Q4_K_XL	8K	158–215
Qwen3.6 35B A3B 34.7B	Tight fit	UD-Q4_K_XL	8K	157–214
GLM 4.7 Flash 29.9B	Tight fit	UD-Q5_K_XL	8K	65–89
Kwaipilot KAT Coder V2.5 Dev 34.7B	Tight fit	Q4_K_M	8K	82–111
Qwen3.8 35B A3B Distill 35.5B	Tight fit	Q4_K_M	8K	165–225
Qwen3 Coder 30B A3B Instruct 30.5B	Tight fit	UD-Q5_K_XL	8K	112–153
Qwen3 30B A3B Instruct 2507 30.5B	Tight fit	UD-Q5_K_XL	8K	112–153
Qwen3 VL 30B A3B Instruct 30.5B	Tight fit	UD-Q5_K_XL	8K	112–153
Qwen3 30B A3B Thinking 2507 30.5B	Tight fit	UD-Q5_K_XL	8K	112–153
Llama 3.3 70B Instruct 70.6B	Tight fit	UD-IQ2_XXS	8K	15–20
DeepSeek R1 Distill Llama 70B 70.6B	Tight fit	UD-IQ2_XXS	8K	15–20
Hermes 4 70B 70.6B	Tight fit	UD-IQ2_XXS	8K	15–20
Nemotron 3 Nano 30B A3B 31.6B	Tight fit	UD-Q4_K_XL	8K	127–173
NVIDIA Nemotron 3 Nano Omni 30B A3B Reasoning 31.6B	Tight fit	UD-Q4_K_S	4K	137–187
gemma 4 26B A4B it 25.2B	Tight fit	UD-Q5_K_XL	8K	61–84
NVIDIA Nemotron 3.5 Lightning 30B A3B 32.9B	Tight fit	IQ4_NL	8K	139–190
Qwen3 Coder Next 79.7B	Tight fit	UD-IQ1_M	8K	74–100
LFM2.5 24B A2B 24.0B	Tight fit	UD-Q6_K_XL	8K	132–180
Qwen3 Next 80B A3B Thinking 79.7B	Tight fit	UD-IQ1_S	4K	262–358
Qwen3 Next 80B A3B Instruct 79.7B	Tight fit	UD-IQ1_S	4K	261–356
GLM 4.7 Flash REAP 23B A3B 23.0B	Tight fit	UD-Q6_K_XL	8K	107–146
gpt oss 20b 20.9B	Runs great	UD-Q8_K_XL	8K	82–112
Qwen2.5 14B Instruct 14.8B	Runs	Q8_0	8K	19–26
NousResearch Hermes 4 14B 14.8B	Runs	Q8_0	8K	19–26
DeepSeek R1 Distill Qwen 14B 14.8B	Runs	Q8_0	8K	19–26

90 more models in the catalog do not fit this default profile, or ranked below the first 40 that do.

Example: Tight fit fully on your NVIDIA A10 at UD-Q5_K_XL, 4K context, about 14–19 tok/s. Weights 21.5 GB plus 0.24 GB of KV cache — 22.3 GB of 22.8 GB usable. Weights shown in the verdict use GiB (the same 1024³ counting as GPU memory).