Hardware

RTX 4000 Ada

RTX 4000 Ada has 20 GB GDDR6 and 360 GB/s. See which open models fit, at which quant, and roughly how fast. Figures below use the default memory (20 GB), 32 GB system RAM, and DDR5-5600 for anything that spills. Updated 2026-10-17.

Memory20 GB GDDR6

Bandwidth360 GB/s

Kindgpu

Year2023

20GB GDDR6, 360 GB/s. Source.

RSS for new models on this device

Models that fit

Model	Verdict	Quant	Context	tok/s
GLM 4 32B 0414 32.6B	Tight fit	Q4_K_S	4K	10–14
Qwen3.8 27B 27.3B	Tight fit	UD-Q4_K_XL	8K	11–15
Qwen3.6 27B 26.9B	Tight fit	UD-Q4_K_XL	8K	11–15
Qwen3.5 27B 26.9B	Tight fit	UD-Q4_K_XL	8K	11–15
Qwen2.5 32B Instruct 32.8B	Tight fit	IQ4_XS	4K	11–14
Qwen3 32B 32.8B	Tight fit	IQ4_XS	4K	11–14
Qwen2.5 Coder 32B Instruct 32.8B	Tight fit	IQ4_XS	4K	11–14
Qwen3 VL 32B Instruct 32.8B	Tight fit	IQ4_XS	4K	11–14
Devstral Small 2 24B Instruct 2512 23.6B	Tight fit	UD-Q5_K_XL	8K	11–15
Mistral Small 3.2 24B Instruct 2506 23.6B	Tight fit	UD-Q5_K_XL	8K	11–15
Magistral Small 2509 23.6B	Tight fit	UD-Q5_K_XL	8K	11–15
Llama 3.3 70B Instruct 70.6B	Tight fit	UD-IQ1_M	4K	11–15
DeepSeek R1 Distill Llama 70B 70.6B	Tight fit	UD-IQ1_M	4K	11–15
Hermes 4 70B 70.6B	Tight fit	UD-IQ1_M	4K	11–15
GLM 4.7 Flash 29.9B	Tight fit	UD-Q4_K_XL	8K	48–65
Qwen3 Coder 30B A3B Instruct 30.5B	Tight fit	UD-Q4_K_XL	8K	78–106
Qwen3 30B A3B Instruct 2507 30.5B	Tight fit	UD-Q4_K_XL	8K	78–106
Qwen3 VL 30B A3B Instruct 30.5B	Tight fit	UD-Q4_K_XL	8K	78–106
Qwen3 30B A3B Thinking 2507 30.5B	Tight fit	UD-Q4_K_XL	8K	78–106
Qwen3.5 35B A3B 34.7B	Tight fit	UD-IQ4_XS	8K	118–161
Qwen3.6 35B A3B 34.7B	Tight fit	UD-IQ4_XS	8K	116–159
Kwaipilot KAT Coder V2.5 Dev 34.7B	Tight fit	IQ4_XS	4K	57–77
DeepSeek R1 Distill Qwen 32B 32.8B	Tight fit	Q3_K_M	8K	11–15
Nemotron 3 Nano 30B A3B 31.6B	Tight fit	IQ4_NL	8K	92–125
LFM2.5 24B A2B 24.0B	Tight fit	UD-Q5_K_XL	8K	91–125
gemma 4 26B A4B it 25.2B	Tight fit	UD-Q4_K_XL	8K	42–57
GLM 4.7 Flash REAP 23B A3B 23.0B	Tight fit	UD-Q5_K_XL	8K	75–103
gpt oss 20b 20.9B	Runs	UD-Q8_K_XL	8K	49–67
Qwen2.5 14B Instruct 14.8B	Tight fit	Q8_0	8K	11–16
Qwen3 14B 14.8B	Tight fit	Q8_0	8K	12–16
NousResearch Hermes 4 14B 14.8B	Tight fit	Q8_0	8K	12–16
DeepSeek R1 Distill Qwen 14B 14.8B	Tight fit	Q8_0	8K	11–16
Qwen2.5 Coder 14B Instruct 128K 14.8B	Tight fit	Q8_0	8K	11–16
Qwen3.8 35B A3B Distill 35.5B	Tight fit	Q3_K_M	8K	119–163
phi 4 14.7B	Tight fit	Q8_0	8K	11–16
Ministral 3 14B Instruct 2512 13.5B	Tight fit	UD-Q8_K_XL	8K	11–15
Ministral 3 14B Reasoning 2512 13.5B	Tight fit	UD-Q8_K_XL	8K	11–15
gemma 4 12b it 11.9B	Runs	UD-Q8_K_XL	8K	12–16
GLM 4.6V Flash 9.4B	Tight fit	BF16	4K	10–14
GLM Z1 9B 0414 9.4B	Tight fit	BF16	4K	10–14

90 more models in the catalog do not fit this default profile, or ranked below the first 40 that do.

Example: Tight fit fully on your RTX 4000 Ada at Q4_K_S, 4K context, about 10–14 tok/s. Weights 17.4 GB plus 0.24 GB of KV cache — 18.1 GB of 18.5 GB usable. Weights shown in the verdict use GiB (the same 1024³ counting as GPU memory).