Hardware

NVIDIA A100 80GB SXM

NVIDIA A100 80GB SXM has 80 GB HBM2e and 2039 GB/s. See which open models fit, at which quant, and roughly how fast. Figures below use the default memory (80 GB), 32 GB system RAM, and DDR5-5600 for anything that spills. Updated 2026-10-24.

Memory80 GB HBM2e

Bandwidth2039 GB/s

Kindgpu

Year2021

SXM4 80GB, 2039 GB/s. Source.

RSS for new models on this device

Models that fit

Model	Verdict	Quant	Context	tok/s
Mistral Medium 3.5 128B 125B	Tight fit	UD-Q4_K_XL	8K	14–19
gpt oss 120b 117B	Runs	UD-Q8_K_XL	8K	91–124
Qwen3.5 122B A10B MTP 125B	Tight fit	UD-Q4_K_XL	4K	175–239
Mistral Small 4 119B 2603 119B	Tight fit	UD-Q4_K_XL	8K	82–112
Llama 4 Scout 17B 16E Instruct 108B	Tight fit	Q5_K_M	8K	63–86
GLM 4.6V 107B	Tight fit	Q5_K_S	8K	63–87
GLM 4.5 Air 110B	Tight fit	UD-Q4_K_XL	8K	70–96
Qwen2.5 72B Instruct 72.7B	Tight fit	Q8_0	4K	14–19
DeepSeek R1 Distill Llama 70B 70.6B	Tight fit	Q8_0	8K	14–20
Llama 3.3 70B Instruct 70.6B	Tight fit	Q8_0	8K	14–20
Hermes 4 70B 70.6B	Tight fit	Q8_0	8K	14–20
NVIDIA Nemotron 3 Super 120B A12B 121B	Runs	IQ4_NL	8K	157–214
Qwen3 Coder Next 79.7B	Tight fit	UD-Q6_K_XL	8K	85–116
Qwen3 Next 80B A3B Instruct 79.7B	Tight fit	UD-Q6_K_XL	8K	331–452
Qwen3 Next 80B A3B Thinking 79.7B	Tight fit	UD-Q6_K_XL	8K	331–452
Step 3.7 Flash 197B	Tight fit	UD-Q2_K_XL	8K	47–64
MiniMax M2 229B	Tight fit	UD-IQ2_XXS	8K	72–99
Qwen3 VL 235B A22B Instruct 235B	Tight fit	UD-IQ1_M	8K	139–189
Qwen3.8 Flash Next 177B	Tight fit	UD-IQ1_M	8K	87–119
Qwen2.5 Coder 32B Instruct 32.8B	Runs great	Q8_0	8K	30–41
Qwen2.5 32B Instruct 32.8B	Runs	F16	8K	17–23
Qwen3 32B 32.8B	Runs	BF16	8K	17–23
Qwen3 VL 32B Instruct 32.8B	Runs	BF16	8K	17–23
DeepSeek R1 Distill Qwen 32B 32.8B	Runs	F16	8K	17–23
GLM 4 32B 0414 32.6B	Runs	BF16	8K	17–23
gemma 4 31B it 30.7B	Runs	BF16	8K	16–22
Qwen3.8 27B 27.3B	Runs	BF16	8K	20–28
Swift Qwen3.8 27B 27.3B	Runs	F16	8K	20–28
gemma 3 27b it 27.0B	Runs	BF16	8K	19–26
medgemma 27b text it 27.0B	Runs	BF16	8K	19–26
Qwen3.6 27B 26.9B	Runs	BF16	8K	21–28
Qwen3.5 27B 26.9B	Runs	BF16	8K	21–28
Kwaipilot KAT Coder V2.5 Dev 34.7B	Tight fit	BF16	8K	88–120
Cyber Tiel Coder 35B A3B GGUF MTP 35.5B	Runs great	Q8_0 (est.)	8K	334–456
Qwen3.8 35B A3B Distill 35.5B	Tight fit	BF16	8K	182–248
Tiel Coder 35B A3B 34.7B	Runs great	UD-Q8_K_XL	8K	321–437
Qwen3.6 35B A3B 34.7B	Tight fit	BF16	8K	182–248
Qwen3.5 35B A3B 34.7B	Tight fit	BF16	8K	182–248
NVIDIA Nemotron 3.5 Lightning 30B A3B 32.9B	Runs	BF16	8K	174–238
Devstral Small 2 24B Instruct 2512 23.6B	Runs	BF16	8K	23–32

105 more models in the catalog do not fit this default profile, or ranked below the first 40 that do.

Example: Tight fit fully on your NVIDIA A100 80GB SXM at UD-Q4_K_XL, 8K context, about 14–19 tok/s. Weights 70.5 GB plus 2.75 GB of KV cache — 74.2 GB of 76.0 GB usable. Weights shown in the verdict use GiB (the same 1024³ counting as GPU memory).