unsloth
gemma 4 E4B it qat
gemma 4 E4B it qat is a 7.46B dense gemma4 model from unsloth. Configured context length is 131,072 tokens. License: apache-2.0. Quant sizes are the GGUF file bytes published on Hugging Face when a file exists.
Parameters7.5B
Context128K
Attentionhybrid
Licenseapache-2.0
42 layers · 2 KV heads · head dim 256 · 35 sliding layers · GGUF repo · config
Quant file sizes
| Quant | File size | Source |
|---|---|---|
| UD-Q4_K_XL | 3.93 GB | Hugging Face file bytes |
| UD-Q2_K_XL | 3.00 GB | Hugging Face file bytes |
On common machines
| Device | Verdict | Quant | Speed |
|---|---|---|---|
| GeForce RTX 4060 Ti 8GB | Runs | UD-Q4_K_XL | 32–44 |
| GeForce RTX 3060 12GB | Runs great | UD-Q4_K_XL | 40–55 |
| GeForce RTX 4060 Ti 16GB | Runs great | UD-Q4_K_XL | 32–44 |
| GeForce RTX 4090 | Runs great | UD-Q4_K_XL | 113–154 |
| GeForce RTX 5090 | Runs great | UD-Q4_K_XL | 200–273 |
| Apple M4 Pro | Runs great | UD-Q4_K_XL | 31–42 |
| Apple M5 | Runs great | UD-Q4_K_XL | 17–23 |
| CPU only · DDR5-5600 | Runs on CPU | UD-Q4_K_XL | 10–14 |
Smallest devices that fit it fully
- GeForce RTX 3050 6GB — Runs, UD-Q4_K_XL, 19–26 tok/s
- GeForce RTX 2060 6GB — Runs, UD-Q4_K_XL, 38–51 tok/s
- GeForce GTX 1660 Super — Runs, UD-Q4_K_XL, 38–51 tok/s
- GeForce GTX 1660 Ti — Runs, UD-Q4_K_XL, 32–44 tok/s
- Intel Arc A380 — Runs, UD-Q4_K_XL, 21–28 tok/s
- GeForce RTX 5060 Ti 8GB — Runs, UD-Q4_K_XL, 50–68 tok/s
Run it
Commands for the GeForce RTX 4060 Ti 16GB estimate (UD-Q4_K_XL).
llama.cpp
llama-server -hf unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL -c 8192 -ngl 42
Ollama
ollama run gemma
LM Studio
lms get unsloth/gemma-4-E4B-it-qat-GGUF # Then load quant UD-Q4_K_XL