Hardware

NVIDIA H100 SXM 80GB

NVIDIA H100 SXM 80GB has 80 GB HBM3 and 3350 GB/s. See which open models fit, at which quant, and roughly how fast. Figures below use the default memory (80 GB), 32 GB system RAM, and DDR5-5600 for anything that spills. Updated 2026-10-17.

Memory80 GB HBM3

Bandwidth3350 GB/s

Kindgpu

Year2022

SXM5 80GB HBM3, 3.35 TB/s. Source.

RSS for new models on this device

Models that fit

Model	Verdict	Quant	Context	tok/s
Mistral Medium 3.5 128B 125B	Tight fit	UD-Q4_K_XL	8K	23–32
Devstral 2 123B Instruct 2512 125B	Tight fit	UD-Q4_K_XL	8K	23–32
gpt oss 120b 117B	Runs	UD-Q8_K_XL	8K	150–204
Qwen3.5 122B A10B MTP 125B	Tight fit	UD-Q4_K_XL	4K	287–392
Mistral Small 4 119B 2603 119B	Tight fit	UD-Q4_K_XL	8K	135–184
Llama 4 Scout 17B 16E Instruct 108B	Tight fit	Q5_K_M	8K	103–141
GLM 4.6V 107B	Tight fit	Q5_K_S	8K	104–142
GLM 4.5 Air 110B	Tight fit	UD-Q4_K_XL	8K	116–158
Qwen2.5 72B Instruct 72.7B	Tight fit	Q8_0	4K	23–32
Llama 3.3 70B Instruct 70.6B	Tight fit	Q8_0	8K	24–32
DeepSeek R1 Distill Llama 70B 70.6B	Tight fit	Q8_0	8K	24–32
Hermes 4 70B 70.6B	Tight fit	Q8_0	8K	24–32
NVIDIA Nemotron 3 Super 120B A12B 121B	Runs	IQ4_NL	8K	258–351
Qwen3 Coder Next 79.7B	Tight fit	UD-Q6_K_XL	8K	140–190
Qwen3 Next 80B A3B Instruct 79.7B	Tight fit	UD-Q6_K_XL	8K	544–742
Qwen3 Next 80B A3B Thinking 79.7B	Tight fit	UD-Q6_K_XL	8K	544–742
Step 3.7 Flash 197B	Tight fit	UD-Q2_K_XL	8K	77–106
MiniMax M2 229B	Tight fit	UD-IQ2_XXS	8K	119–162
Qwen3.8 Flash Next 177B	Tight fit	UD-IQ1_M	8K	144–196
Qwen2.5 Coder 32B Instruct 32.8B	Runs great	Q8_0	8K	50–68
Qwen2.5 32B Instruct 32.8B	Runs	F16	8K	27–37
Qwen3 32B 32.8B	Runs	BF16	8K	27–37
DeepSeek R1 Distill Qwen 32B 32.8B	Runs	F16	8K	27–37
Qwen3 VL 32B Instruct 32.8B	Runs	BF16	8K	27–37
GLM 4 32B 0414 32.6B	Runs	BF16	8K	28–38
gemma 4 31B it 30.7B	Runs	BF16	8K	27–36
Qwen3.8 27B 27.3B	Runs	BF16	8K	33–46
gemma 3 27b it 27.0B	Runs	BF16	8K	32–43
medgemma 27b text it 27.0B	Runs	BF16	8K	32–43
Qwen3.6 27B 26.9B	Runs	BF16	8K	34–46
Qwen3.5 27B 26.9B	Runs	BF16	8K	34–46
Kwaipilot KAT Coder V2.5 Dev 34.7B	Tight fit	BF16	8K	145–197
Qwen3.8 35B A3B Distill 35.5B	Tight fit	BF16	8K	298–407
Qwen3.6 35B A3B 34.7B	Tight fit	BF16	8K	299–407
Qwen3.5 35B A3B 34.7B	Tight fit	BF16	8K	299–407
NVIDIA Nemotron 3.5 Lightning 30B A3B 32.9B	Runs	BF16	8K	286–390
Devstral Small 2 24B Instruct 2512 23.6B	Runs	BF16	8K	38–52
Mistral Small 3.2 24B Instruct 2506 23.6B	Runs	BF16	8K	38–52
Magistral Small 2509 23.6B	Runs	BF16	8K	38–52
NVIDIA Nemotron 3 Nano Omni 30B A3B Reasoning 31.6B	Runs great	UD-Q8_K_XL	8K	485–661

90 more models in the catalog do not fit this default profile, or ranked below the first 40 that do.

Example: Tight fit fully on your NVIDIA H100 SXM 80GB at UD-Q4_K_XL, 8K context, about 23–32 tok/s. Weights 70.5 GB plus 2.75 GB of KV cache — 74.2 GB of 76.0 GB usable. Weights shown in the verdict use GiB (the same 1024³ counting as GPU memory).