Hardware

NVIDIA H200 SXM

NVIDIA H200 SXM has 141 GB HBM3e and 4800 GB/s. See which open models fit, at which quant, and roughly how fast. Figures below use the default memory (141 GB), 32 GB system RAM, and DDR5-5600 for anything that spills. Updated 2026-10-17.

Memory141 GB HBM3e

Bandwidth4800 GB/s

Kindgpu

Year2024

141GB HBM3e, 4.8 TB/s. Source.

RSS for new models on this device

Models that fit

Model	Verdict	Quant	Context	tok/s
DeepSeek V4 Flash 0731 284B	Tight fit	IQ4_NL	8K	108–147
DeepSeek V4 Flash Vision Exp 284B	Tight fit	UD-IQ4_XS	8K	108–147
MiniMax M2 229B	Tight fit	UD-Q4_K_XL	8K	102–139
Qwen3 235B A22B Instruct 2507 235B	Tight fit	UD-Q4_K_XL	8K	187–254
Qwen3 235B A22B Thinking 2507 235B	Tight fit	UD-Q4_K_XL	8K	187–255
MiMo V2 Flash 309B	Tight fit	UD-Q3_K_XL	8K	101–138
Step 3.7 Flash 197B	Tight fit	UD-Q4_K_XL	8K	78–106
Mistral Medium 3.5 128B 125B	Tight fit	Q8_0	8K	19–27
Devstral 2 123B Instruct 2512 125B	Tight fit	Q8_0	8K	19–27
Qwen3.8 Flash Next 177B	Runs	UD-Q4_K_XL	8K	138–189
GLM 5.3 Flash 321B	Runs	UD-IQ3_XXS	8K	123–168
GLM 4.6 357B	Tight fit	UD-Q2_K_XL	8K	88–120
Llama 4 Maverick 17B 128E Instruct 401B	Tight fit	UD-IQ2_XXS	8K	115–157
Qwen3.5 122B A10B MTP 125B	Tight fit	UD-Q8_K_XL	8K	239–326
Mistral Small 4 119B 2603 119B	Tight fit	UD-Q8_K_XL	8K	109–149
gpt oss 120b 117B	Runs great	UD-Q8_K_XL	8K	215–293
NVIDIA Nemotron 3 Super 120B A12B 121B	Tight fit	UD-Q8_K_XL	8K	190–259
Qwen3.5 397B A17B 396B	Runs	UD-IQ2_XXS	8K	510–695
GLM 4.5 Air 110B	Tight fit	UD-Q8_K_XL	8K	92–125
Llama 4 Scout 17B 16E Instruct 108B	Tight fit	UD-Q8_K_XL	8K	92–125
GLM 4.6V 107B	Tight fit	UD-Q8_K_XL	8K	95–129
Qwen2.5 72B Instruct 72.7B	Runs great	Q8_0	8K	33–45
Llama 3.3 70B Instruct 70.6B	Runs great	UD-Q8_K_XL	8K	31–43
DeepSeek R1 Distill Llama 70B 70.6B	Runs great	UD-Q8_K_XL	8K	31–43
Hermes 4 70B 70.6B	Runs great	UD-Q8_K_XL	8K	31–43
Qwen3 Coder Next 79.7B	Runs	UD-Q8_K_XL	8K	171–233
Qwen3 Next 80B A3B Instruct 79.7B	Runs	UD-Q8_K_XL	8K	613–835
Qwen3 Next 80B A3B Thinking 79.7B	Runs	UD-Q8_K_XL	8K	613–835
Qwen2.5 Coder 32B Instruct 32.8B	Runs great	Q8_0	8K	71–97
Qwen2.5 32B Instruct 32.8B	Runs great	F16	8K	39–53
Qwen3 32B 32.8B	Runs great	BF16	8K	39–53
DeepSeek R1 Distill Qwen 32B 32.8B	Runs great	F16	8K	39–53
Qwen3 VL 32B Instruct 32.8B	Runs great	BF16	8K	39–53
GLM 4 32B 0414 32.6B	Runs great	BF16	8K	40–55
gemma 4 31B it 30.7B	Runs great	BF16	8K	38–52
Qwen3.8 27B 27.3B	Runs great	BF16	8K	48–65
gemma 3 27b it 27.0B	Runs great	BF16	8K	45–62
medgemma 27b text it 27.0B	Runs great	BF16	8K	45–62
Qwen3.6 27B 26.9B	Runs great	BF16	8K	49–66
Qwen3.5 27B 26.9B	Runs great	BF16	8K	49–66

90 more models in the catalog do not fit this default profile, or ranked below the first 40 that do.

Example: Tight fit fully on your NVIDIA H200 SXM at IQ4_NL, 8K context, about 108–147 tok/s. Weights 127.3 GB plus 0.67 GB of KV cache — 129.5 GB of 133.9 GB usable. Up to 16K context still fits. Weights shown in the verdict use GiB (the same 1024³ counting as GPU memory).