Local LLMs, without the spreadsheet
Pick a GPU, an Apple chip, or a CPU. GPUfit ranks 130 open models with a plain-English verdict, the best quant that fits, a practical context length, and a tokens-per-second range. The profile stays in this browser. Catalog updated 2026-10-17. 120 devices, sourced specs.
Example machine
16 GB GDDR6 · 288 GB/s · 32 GB system RAM · pick yours to replace this
The largest checkpoints that still fit fully at about 10 tok/s or faster. Open the Small filter for the tiny ones.
130 models · 92 fit fully on this profile. Estimates assume llama.cpp, FP16 KV cache, and a bandwidth range.
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-IQ4_XS, 8K context, about 11–15 tok/s. Weights 13.3 GB plus 0.57 GB of KV cache — 14.3 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3.8-27B-GGUF:UD-IQ4_XS -c 8192 -ngl 64
ollama run qwen3.8
lms get unsloth/Qwen3.8-27B-GGUF # Then load quant UD-IQ4_XS
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q4_K_M, 4K context, about 11–14 tok/s. Weights 13.3 GB plus 0.63 GB of KV cache — 14.4 GB of 14.7 GB usable.
llama-server -hf unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF:Q4_K_M -c 4096 -ngl 40
ollama run devstral
lms get unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF # Then load quant Q4_K_M
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q4_K_M, 4K context, about 11–14 tok/s. Weights 13.3 GB plus 0.63 GB of KV cache — 14.4 GB of 14.7 GB usable.
llama-server -hf unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:Q4_K_M -c 4096 -ngl 40
ollama run mistral
lms get unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF # Then load quant Q4_K_M
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q4_K_M, 4K context, about 11–14 tok/s. Weights 13.3 GB plus 0.63 GB of KV cache — 14.4 GB of 14.7 GB usable.
llama-server -hf unsloth/Magistral-Small-2509-GGUF:Q4_K_M -c 4096 -ngl 40
ollama run magistral
lms get unsloth/Magistral-Small-2509-GGUF # Then load quant Q4_K_M
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 39–54 tok/s. Weights 12.3 GB plus 0.19 GB of KV cache — 12.9 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/gpt-oss-20b-GGUF:UD-Q8_K_XL -c 8192 -ngl 24
No Ollama library match for this repo.
lms get unsloth/gpt-oss-20b-GGUF # Then load quant UD-Q8_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 4K context, about 11–15 tok/s. Weights 13.5 GB plus 0.32 GB of KV cache — 14.2 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3.6-27B-GGUF:UD-Q3_K_XL -c 4096 -ngl 64
ollama run qwen3.6
lms get unsloth/Qwen3.6-27B-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 4K context, about 11–15 tok/s. Weights 13.4 GB plus 0.32 GB of KV cache — 14.2 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3.5-27B-GGUF:UD-Q3_K_XL -c 4096 -ngl 64
ollama run qwen3.5
lms get unsloth/Qwen3.5-27B-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q4_K_XL, 8K context, about 69–94 tok/s. Weights 13.3 GB plus 0.41 GB of KV cache — 14.1 GB of 14.7 GB usable.
llama-server -hf unsloth/GLM-4.7-Flash-REAP-23B-A3B-GGUF:UD-Q4_K_XL -c 8192 -ngl 47
No Ollama library match for this repo.
lms get unsloth/GLM-4.7-Flash-REAP-23B-A3B-GGUF # Then load quant UD-Q4_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q3_K_S, 8K context, about 11–15 tok/s. Weights 13.4 GB plus 0.48 GB of KV cache — 14.3 GB of 14.7 GB usable.
llama-server -hf unsloth/GLM-4-32B-0414-GGUF:Q3_K_S -c 8192 -ngl 61
No Ollama library match for this repo.
lms get unsloth/GLM-4-32B-0414-GGUF # Then load quant Q3_K_S
Runs fully on your GeForce RTX 4060 Ti 16GB at Q5_K_M, 8K context, about 13–18 tok/s. Weights 9.79 GB plus 1.50 GB of KV cache — 11.7 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf bartowski/Qwen2.5-14B-Instruct-GGUF:Q5_K_M -c 8192 -ngl 48
ollama run qwen2.5
lms get bartowski/Qwen2.5-14B-Instruct-GGUF # Then load quant Q5_K_M
Runs fully on your GeForce RTX 4060 Ti 16GB at Q5_K_M, 8K context, about 13–18 tok/s. Weights 9.79 GB plus 1.25 GB of KV cache — 11.4 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf bartowski/NousResearch_Hermes-4-14B-GGUF:Q5_K_M -c 8192 -ngl 40
No Ollama library match for this repo.
lms get bartowski/NousResearch_Hermes-4-14B-GGUF # Then load quant Q5_K_M
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q6_K, 8K context, about 12–16 tok/s. Weights 11.3 GB plus 1.50 GB of KV cache — 13.2 GB of 14.7 GB usable.
llama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q6_K -c 8192 -ngl 48
No Ollama library match for this repo.
lms get unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF # Then load quant Q6_K
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q6_K, 8K context, about 12–16 tok/s. Weights 11.3 GB plus 1.50 GB of KV cache — 13.2 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF:Q6_K -c 8192 -ngl 48
ollama run qwen2.5
lms get unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF # Then load quant Q6_K
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q6_K_XL, 8K context, about 11–15 tok/s. Weights 12.4 GB plus 1.25 GB of KV cache — 14.0 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3-14B-GGUF:UD-Q6_K_XL -c 8192 -ngl 40
ollama run qwen3
lms get unsloth/Qwen3-14B-GGUF # Then load quant UD-Q6_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q6_K, 8K context, about 12–16 tok/s. Weights 11.2 GB plus 1.56 GB of KV cache — 13.2 GB of 14.7 GB usable.
llama-server -hf unsloth/phi-4-GGUF:Q6_K -c 8192 -ngl 40
ollama run phi4
lms get unsloth/phi-4-GGUF # Then load quant Q6_K
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-IQ4_XS, 8K context, about 38–52 tok/s. Weights 12.7 GB plus 0.51 GB of KV cache — 13.6 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-IQ4_XS -c 8192 -ngl 30
ollama run gemma
lms get unsloth/gemma-4-26B-A4B-it-GGUF # Then load quant UD-IQ4_XS
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-IQ3_XXS, 4K context, about 11–15 tok/s. Weights 12.1 GB plus 1.00 GB of KV cache — 13.5 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3-32B-GGUF:UD-IQ3_XXS -c 4096 -ngl 64
ollama run qwen3
lms get unsloth/Qwen3-32B-GGUF # Then load quant UD-IQ3_XXS
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-IQ3_XXS, 4K context, about 11–15 tok/s. Weights 12.1 GB plus 1.00 GB of KV cache — 13.5 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3-VL-32B-Instruct-GGUF:UD-IQ3_XXS -c 4096 -ngl 64
ollama run qwen3
lms get unsloth/Qwen3-VL-32B-Instruct-GGUF # Then load quant UD-IQ3_XXS
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 8K context, about 47–64 tok/s. Weights 12.8 GB plus 0.41 GB of KV cache — 13.7 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/GLM-4.7-Flash-GGUF:UD-Q3_K_XL -c 8192 -ngl 47
ollama run glm-4.7-flash
lms get unsloth/GLM-4.7-Flash-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-IQ3_S, 8K context, about 118–161 tok/s. Weights 12.7 GB plus 0.19 GB of KV cache — 13.3 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/Qwen3.5-35B-A3B-GGUF:UD-IQ3_S -c 8192 -ngl 40
ollama run qwen3.5
lms get unsloth/Qwen3.5-35B-A3B-GGUF # Then load quant UD-IQ3_S
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-IQ3_S, 8K context, about 117–160 tok/s. Weights 12.7 GB plus 0.19 GB of KV cache — 13.4 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-IQ3_S -c 8192 -ngl 40
ollama run qwen3.6
lms get unsloth/Qwen3.6-35B-A3B-GGUF # Then load quant UD-IQ3_S
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q6_K_XL, 8K context, about 12–16 tok/s. Weights 11.3 GB plus 1.25 GB of KV cache — 13.0 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/Ministral-3-14B-Instruct-2512-GGUF:UD-Q6_K_XL -c 8192 -ngl 40
No Ollama library match for this repo.
lms get unsloth/Ministral-3-14B-Instruct-2512-GGUF # Then load quant UD-Q6_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q6_K_XL, 8K context, about 12–16 tok/s. Weights 11.3 GB plus 1.25 GB of KV cache — 13.0 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/Ministral-3-14B-Reasoning-2512-GGUF:UD-Q6_K_XL -c 8192 -ngl 40
No Ollama library match for this repo.
lms get unsloth/Ministral-3-14B-Reasoning-2512-GGUF # Then load quant UD-Q6_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 8K context, about 73–100 tok/s. Weights 12.9 GB plus 0.75 GB of KV cache — 14.0 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:UD-Q3_K_XL -c 8192 -ngl 48
ollama run qwen3
lms get unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 8K context, about 73–100 tok/s. Weights 12.9 GB plus 0.75 GB of KV cache — 14.1 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF:UD-Q3_K_XL -c 8192 -ngl 48
ollama run qwen3
lms get unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 8K context, about 73–100 tok/s. Weights 12.9 GB plus 0.75 GB of KV cache — 14.1 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3-VL-30B-A3B-Instruct-GGUF:UD-Q3_K_XL -c 8192 -ngl 48
ollama run qwen3
lms get unsloth/Qwen3-VL-30B-A3B-Instruct-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 8K context, about 73–100 tok/s. Weights 12.9 GB plus 0.75 GB of KV cache — 14.1 GB of 14.7 GB usable.
llama-server -hf unsloth/Qwen3-30B-A3B-Thinking-2507-GGUF:UD-Q3_K_XL -c 8192 -ngl 48
ollama run qwen3
lms get unsloth/Qwen3-30B-A3B-Thinking-2507-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at IQ3_XXS, 8K context, about 11–14 tok/s. Weights 12.0 GB plus 2.00 GB of KV cache — 14.4 GB of 14.7 GB usable.
llama-server -hf bartowski/Qwen2.5-Coder-32B-Instruct-GGUF:IQ3_XXS -c 8192 -ngl 64
ollama run qwen2.5
lms get bartowski/Qwen2.5-Coder-32B-Instruct-GGUF # Then load quant IQ3_XXS
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 8K context, about 100–136 tok/s. Weights 10.2 GB plus 0.63 GB of KV cache — 11.2 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/LFM2.5-24B-A2B-GGUF:UD-Q3_K_XL -c 8192 -ngl 40
ollama run lfm2.5
lms get unsloth/LFM2.5-24B-A2B-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at IQ3_XXS, 4K context, about 57–78 tok/s. Weights 13.8 GB plus 0.11 GB of KV cache — 14.4 GB of 14.7 GB usable.
llama-server -hf bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF:IQ3_XXS -c 4096 -ngl 40
No Ollama library match for this repo.
lms get bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF # Then load quant IQ3_XXS
Runs fully on your GeForce RTX 4060 Ti 16GB at IQ2_S, 8K context, about 13–17 tok/s. Weights 9.67 GB plus 2.00 GB of KV cache — 12.1 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf bartowski/Qwen2.5-32B-Instruct-GGUF:IQ2_S -c 8192 -ngl 64
ollama run qwen2.5
lms get bartowski/Qwen2.5-32B-Instruct-GGUF # Then load quant IQ2_S
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–17 tok/s. Weights 11.2 GB plus 0.31 GB of KV cache — 12.0 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/GLM-4.6V-Flash-GGUF:UD-Q8_K_XL -c 8192 -ngl 40
No Ollama library match for this repo.
lms get unsloth/GLM-4.6V-Flash-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–17 tok/s. Weights 11.2 GB plus 0.31 GB of KV cache — 12.0 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/GLM-Z1-9B-0414-GGUF:UD-Q8_K_XL -c 8192 -ngl 40
No Ollama library match for this repo.
lms get unsloth/GLM-Z1-9B-0414-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–17 tok/s. Weights 11.2 GB plus 0.31 GB of KV cache — 12.0 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/GLM-4-9B-0414-GGUF:UD-Q8_K_XL -c 8192 -ngl 40
No Ollama library match for this repo.
lms get unsloth/GLM-4-9B-0414-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–17 tok/s. Weights 11.2 GB plus 0.31 GB of KV cache — 12.0 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/GLM-4.1V-9B-Thinking-GGUF:UD-Q8_K_XL -c 8192 -ngl 40
No Ollama library match for this repo.
lms get unsloth/GLM-4.1V-9B-Thinking-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at Q8_0, 8K context, about 16–22 tok/s. Weights 8.89 GB plus 0.27 GB of KV cache — 9.55 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q8_0 -c 8192 -ngl 32
No Ollama library match for this repo.
lms get bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF # Then load quant Q8_0
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 12–16 tok/s. Weights 12.1 GB plus 0.27 GB of KV cache — 12.8 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Qwen3.5-9B-GGUF:UD-Q8_K_XL -c 8192 -ngl 32
ollama run qwen3.5
lms get unsloth/Qwen3.5-9B-GGUF # Then load quant UD-Q8_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q2_K, 8K context, about 118–162 tok/s. Weights 12.9 GB plus 0.19 GB of KV cache — 13.5 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf empero-ai/Qwen3.8-35B-A3B-Distill-GGUF:Q2_K -c 8192 -ngl 40
ollama run qwen3.8
lms get empero-ai/Qwen3.8-35B-A3B-Distill-GGUF # Then load quant Q2_K
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–18 tok/s. Weights 10.3 GB plus 1.06 GB of KV cache — 11.8 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/Ministral-3-8B-Instruct-2512-GGUF:UD-Q8_K_XL -c 8192 -ngl 34
No Ollama library match for this repo.
lms get unsloth/Ministral-3-8B-Instruct-2512-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–18 tok/s. Weights 10.3 GB plus 1.06 GB of KV cache — 11.8 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/Ministral-3-8B-Reasoning-2512-GGUF:UD-Q8_K_XL -c 8192 -ngl 34
No Ollama library match for this repo.
lms get unsloth/Ministral-3-8B-Reasoning-2512-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–18 tok/s. Weights 10.1 GB plus 1.13 GB of KV cache — 11.6 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/Qwen3-VL-8B-Instruct-GGUF:UD-Q8_K_XL -c 8192 -ngl 36
ollama run qwen3
lms get unsloth/Qwen3-VL-8B-Instruct-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–18 tok/s. Weights 10.1 GB plus 1.13 GB of KV cache — 11.6 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF:UD-Q8_K_XL -c 8192 -ngl 36
No Ollama library match for this repo.
lms get unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–18 tok/s. Weights 10.1 GB plus 1.13 GB of KV cache — 11.6 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/Qwen3-8B-GGUF:UD-Q8_K_XL -c 8192 -ngl 36
ollama run qwen3
lms get unsloth/Qwen3-8B-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 13–18 tok/s. Weights 10.1 GB plus 1.13 GB of KV cache — 11.6 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/Qwen3-VL-8B-Thinking-GGUF:UD-Q8_K_XL -c 8192 -ngl 36
ollama run qwen3
lms get unsloth/Qwen3-VL-8B-Thinking-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 14–19 tok/s. Weights 9.85 GB plus 1.00 GB of KV cache — 11.2 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF:UD-Q8_K_XL -c 8192 -ngl 32
No Ollama library match for this repo.
lms get unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 14–19 tok/s. Weights 9.85 GB plus 1.00 GB of KV cache — 11.2 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Llama-3.1-8B-Instruct-GGUF:UD-Q8_K_XL -c 8192 -ngl 32
No Ollama library match for this repo.
lms get unsloth/Llama-3.1-8B-Instruct-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 14–18 tok/s. Weights 9.89 GB plus 1.00 GB of KV cache — 11.3 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Apertus-8B-Instruct-2509-GGUF:UD-Q8_K_XL -c 8192 -ngl 32
No Ollama library match for this repo.
lms get unsloth/Apertus-8B-Instruct-2509-GGUF # Then load quant UD-Q8_K_XL
Runs great fully on your GeForce RTX 4060 Ti 16GB at Q8_0, 8K context, about 18–25 tok/s. Weights 7.54 GB plus 0.44 GB of KV cache — 8.36 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf bartowski/Qwen2.5-7B-Instruct-GGUF:Q8_0 -c 8192 -ngl 28
ollama run qwen2.5
lms get bartowski/Qwen2.5-7B-Instruct-GGUF # Then load quant Q8_0
Runs great fully on your GeForce RTX 4060 Ti 16GB at Q8_0, 8K context, about 18–25 tok/s. Weights 7.54 GB plus 0.44 GB of KV cache — 8.36 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q8_0 -c 8192 -ngl 28
No Ollama library match for this repo.
lms get unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF # Then load quant Q8_0
Runs great fully on your GeForce RTX 4060 Ti 16GB at Q8_0, 8K context, about 18–25 tok/s. Weights 7.54 GB plus 0.44 GB of KV cache — 8.36 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/Qwen2.5-Coder-7B-Instruct-128K-GGUF:Q8_0 -c 8192 -ngl 28
ollama run qwen2.5
lms get unsloth/Qwen2.5-Coder-7B-Instruct-128K-GGUF # Then load quant Q8_0
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 15–20 tok/s. Weights 9.49 GB plus 0.44 GB of KV cache — 10.3 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/Qwen2.5-VL-7B-Instruct-GGUF:UD-Q8_K_XL -c 8192 -ngl 28
ollama run qwen2.5
lms get unsloth/Qwen2.5-VL-7B-Instruct-GGUF # Then load quant UD-Q8_K_XL
Runs great fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 17–23 tok/s. Weights 8.11 GB plus 0.14 GB of KV cache — 8.64 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf unsloth/gemma-4-E4B-it-GGUF:UD-Q8_K_XL -c 8192 -ngl 42
ollama run gemma
lms get unsloth/gemma-4-E4B-it-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 12–16 tok/s. Weights 8.68 GB plus 2.50 GB of KV cache — 11.6 GB of 14.7 GB usable. Up to 16K context still fits.
llama-server -hf unsloth/Olmo-3-7B-Instruct-GGUF:UD-Q8_K_XL -c 8192 -ngl 32
No Ollama library match for this repo.
lms get unsloth/Olmo-3-7B-Instruct-GGUF # Then load quant UD-Q8_K_XL
Runs great fully on your GeForce RTX 4060 Ti 16GB at UD-Q4_K_XL, 8K context, about 32–44 tok/s. Weights 3.93 GB plus 0.14 GB of KV cache — 4.41 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL -c 8192 -ngl 42
ollama run gemma
lms get unsloth/gemma-4-E4B-it-qat-GGUF # Then load quant UD-Q4_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 105–143 tok/s. Weights 8.70 GB plus 0.38 GB of KV cache — 9.46 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/LFM2.5-8B-A1B-GGUF:UD-Q8_K_XL -c 8192 -ngl 24
ollama run lfm2.5
lms get unsloth/LFM2.5-8B-A1B-GGUF # Then load quant UD-Q8_K_XL
Runs fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 16–22 tok/s. Weights 8.67 GB plus 0.07 GB of KV cache — 9.13 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf unsloth/gemma-4-E2B-it-GGUF:BF16 -c 8192 -ngl 35
ollama run gemma
lms get unsloth/gemma-4-E2B-it-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 18–25 tok/s. Weights 7.85 GB plus 0.27 GB of KV cache — 8.50 GB of 14.7 GB usable. Up to 200K context still fits.
llama-server -hf unsloth/Qwen3.5-4B-GGUF:BF16 -c 8192 -ngl 32
ollama run qwen3.5
lms get unsloth/Qwen3.5-4B-GGUF # Then load quant BF16
Runs fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 17–23 tok/s. Weights 7.50 GB plus 1.13 GB of KV cache — 9.00 GB of 14.7 GB usable. Up to 40K context still fits.
llama-server -hf unsloth/Qwen3-4B-GGUF:BF16 -c 8192 -ngl 36
ollama run qwen3
lms get unsloth/Qwen3-4B-GGUF # Then load quant BF16
Runs fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 17–23 tok/s. Weights 7.50 GB plus 1.13 GB of KV cache — 9.00 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Qwen3-VL-4B-Instruct-GGUF:BF16 -c 8192 -ngl 36
ollama run qwen3
lms get unsloth/Qwen3-VL-4B-Instruct-GGUF # Then load quant BF16
Runs fully on your GeForce RTX 4060 Ti 16GB at F16, 8K context, about 17–23 tok/s. Weights 7.50 GB plus 1.13 GB of KV cache — 9.00 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Qwen3-4B-Thinking-2507-GGUF:F16 -c 8192 -ngl 36
ollama run qwen3
lms get unsloth/Qwen3-4B-Thinking-2507-GGUF # Then load quant F16
Runs fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 17–23 tok/s. Weights 7.50 GB plus 1.13 GB of KV cache — 9.00 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Qwen3-VL-4B-Thinking-GGUF:BF16 -c 8192 -ngl 36
ollama run qwen3
lms get unsloth/Qwen3-VL-4B-Thinking-GGUF # Then load quant BF16
Runs fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 17–23 tok/s. Weights 7.41 GB plus 1.31 GB of KV cache — 9.10 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF:BF16 -c 8192 -ngl 42
No Ollama library match for this repo.
lms get unsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 18–24 tok/s. Weights 7.23 GB plus 0.30 GB of KV cache — 7.90 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf unsloth/gemma-3-4b-it-GGUF:BF16 -c 8192 -ngl 34
ollama run gemma
lms get unsloth/gemma-3-4b-it-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 18–24 tok/s. Weights 7.23 GB plus 0.30 GB of KV cache — 7.90 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf unsloth/medgemma-4b-it-GGUF:BF16 -c 8192 -ngl 34
ollama run medgemma
lms get unsloth/medgemma-4b-it-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 18–25 tok/s. Weights 7.15 GB plus 1.00 GB of KV cache — 8.52 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Phi-4-mini-instruct-GGUF:BF16 -c 8192 -ngl 32
ollama run phi
lms get unsloth/Phi-4-mini-instruct-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 18–25 tok/s. Weights 7.15 GB plus 1.00 GB of KV cache — 8.52 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/Phi-4-mini-reasoning-GGUF:BF16 -c 8192 -ngl 32
ollama run phi
lms get unsloth/Phi-4-mini-reasoning-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at Q8_0, 8K context, about 35–48 tok/s. Weights 3.40 GB plus 0.81 GB of KV cache — 4.55 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf lmstudio-community/Ministral-3-3B-Instruct-2512-GGUF:Q8_0 -c 8192 -ngl 26
No Ollama library match for this repo.
lms get lmstudio-community/Ministral-3-3B-Instruct-2512-GGUF # Then load quant Q8_0
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 20–28 tok/s. Weights 6.39 GB plus 0.81 GB of KV cache — 7.57 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/Ministral-3-3B-Reasoning-2512-GGUF:BF16 -c 8192 -ngl 26
No Ollama library match for this repo.
lms get unsloth/Ministral-3-3B-Reasoning-2512-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 25–33 tok/s. Weights 5.95 GB plus 0.13 GB of KV cache — 6.44 GB of 14.7 GB usable. Up to 1024K context still fits.
llama-server -hf unsloth/granite-4.0-h-micro-GGUF:BF16 -c 8192 -ngl 40
No Ollama library match for this repo.
lms get unsloth/granite-4.0-h-micro-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 21–29 tok/s. Weights 5.99 GB plus 0.88 GB of KV cache — 7.23 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/Llama-3.2-3B-Instruct-GGUF:BF16 -c 8192 -ngl 28
No Ollama library match for this repo.
lms get unsloth/Llama-3.2-3B-Instruct-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at F16, 8K context, about 24–33 tok/s. Weights 5.75 GB plus 0.28 GB of KV cache — 6.39 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf bartowski/Qwen2.5-3B-Instruct-GGUF:F16 -c 8192 -ngl 36
ollama run qwen2.5
lms get bartowski/Qwen2.5-3B-Instruct-GGUF # Then load quant F16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 24–33 tok/s. Weights 5.75 GB plus 0.28 GB of KV cache — 6.39 GB of 14.7 GB usable. Up to 125K context still fits.
llama-server -hf unsloth/Qwen2.5-VL-3B-Instruct-GGUF:BF16 -c 8192 -ngl 36
ollama run qwen2.5
lms get unsloth/Qwen2.5-VL-3B-Instruct-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 23–32 tok/s. Weights 5.74 GB plus 0.56 GB of KV cache — 6.66 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/SmolLM3-3B-GGUF:BF16 -c 8192 -ngl 36
No Ollama library match for this repo.
lms get unsloth/SmolLM3-3B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 27–37 tok/s. Weights 5.03 GB plus 0.47 GB of KV cache — 5.85 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf LiquidAI/LFM2.5-2.6B-GGUF:BF16 -c 8192 -ngl 30
ollama run lfm2.5
lms get LiquidAI/LFM2.5-2.6B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 27–37 tok/s. Weights 5.03 GB plus 0.47 GB of KV cache — 5.85 GB of 14.7 GB usable. Up to 125K context still fits.
llama-server -hf unsloth/LFM2.5-VL-3B-GGUF:BF16 -c 8192 -ngl 30
ollama run lfm2.5
lms get unsloth/LFM2.5-VL-3B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 41–56 tok/s. Weights 3.52 GB plus 0.10 GB of KV cache — 3.95 GB of 14.7 GB usable. Up to 256K context still fits.
llama-server -hf unsloth/Qwen3.5-2B-GGUF:BF16 -c 8192 -ngl 24
ollama run qwen3.5
lms get unsloth/Qwen3.5-2B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 42–57 tok/s. Weights 3.32 GB plus 0.22 GB of KV cache — 3.87 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-1.5B-GGUF:BF16 -c 8192 -ngl 28
No Ollama library match for this repo.
lms get unsloth/DeepSeek-R1-Distill-Qwen-1.5B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 36–49 tok/s. Weights 3.21 GB plus 0.88 GB of KV cache — 4.42 GB of 14.7 GB usable. Up to 40K context still fits.
llama-server -hf unsloth/Qwen3-1.7B-GGUF:BF16 -c 8192 -ngl 28
ollama run qwen3
lms get unsloth/Qwen3-1.7B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 36–49 tok/s. Weights 3.21 GB plus 0.88 GB of KV cache — 4.42 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/Qwen3-VL-2B-Instruct-GGUF:BF16 -c 8192 -ngl 28
ollama run qwen3
lms get unsloth/Qwen3-VL-2B-Instruct-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 36–49 tok/s. Weights 3.21 GB plus 0.88 GB of KV cache — 4.42 GB of 14.7 GB usable. Up to 64K context still fits.
llama-server -hf unsloth/Qwen3-VL-2B-Thinking-GGUF:BF16 -c 8192 -ngl 28
ollama run qwen3
lms get unsloth/Qwen3-VL-2B-Thinking-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at F16, 8K context, about 48–65 tok/s. Weights 2.88 GB plus 0.22 GB of KV cache — 3.43 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf bartowski/Qwen2.5-1.5B-Instruct-GGUF:F16 -c 8192 -ngl 28
ollama run qwen2.5
lms get bartowski/Qwen2.5-1.5B-Instruct-GGUF # Then load quant F16
Runs great fully on your GeForce RTX 4060 Ti 16GB at F16, 8K context, about 48–65 tok/s. Weights 2.88 GB plus 0.22 GB of KV cache — 3.43 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF:F16 -c 8192 -ngl 28
ollama run qwen2.5
lms get bartowski/Qwen2.5-Coder-1.5B-Instruct-GGUF # Then load quant F16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 58–79 tok/s. Weights 2.31 GB plus 0.25 GB of KV cache — 2.88 GB of 14.7 GB usable. Up to 128K context still fits.
llama-server -hf unsloth/Llama-3.2-1B-Instruct-GGUF:BF16 -c 8192 -ngl 16
No Ollama library match for this repo.
lms get unsloth/Llama-3.2-1B-Instruct-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 61–83 tok/s. Weights 2.18 GB plus 0.25 GB of KV cache — 2.75 GB of 14.7 GB usable. Up to 125K context still fits.
llama-server -hf unsloth/LFM2.5-1.2B-Instruct-GGUF:BF16 -c 8192 -ngl 16
ollama run lfm2.5
lms get unsloth/LFM2.5-1.2B-Instruct-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 71–97 tok/s. Weights 1.87 GB plus 0.04 GB of KV cache — 2.23 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf unsloth/gemma-3-1b-it-GGUF:BF16 -c 8192 -ngl 26
ollama run gemma
lms get unsloth/gemma-3-1b-it-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 98–134 tok/s. Weights 1.41 GB plus 0.10 GB of KV cache — 1.83 GB of 14.7 GB usable. Up to 256K context still fits.
llama-server -hf unsloth/Qwen3.5-0.8B-GGUF:BF16 -c 8192 -ngl 24
ollama run qwen3.5
lms get unsloth/Qwen3.5-0.8B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at BF16, 8K context, about 74–101 tok/s. Weights 1.12 GB plus 0.88 GB of KV cache — 2.30 GB of 14.7 GB usable. Up to 40K context still fits.
llama-server -hf unsloth/Qwen3-0.6B-GGUF:BF16 -c 8192 -ngl 28
ollama run qwen3
lms get unsloth/Qwen3-0.6B-GGUF # Then load quant BF16
Runs great fully on your GeForce RTX 4060 Ti 16GB at F16, 8K context, about 145–197 tok/s. Weights 0.93 GB plus 0.09 GB of KV cache — 1.33 GB of 14.7 GB usable. Up to 32K context still fits.
llama-server -hf bartowski/Qwen2.5-0.5B-Instruct-GGUF:F16 -c 8192 -ngl 24
ollama run qwen2.5
lms get bartowski/Qwen2.5-0.5B-Instruct-GGUF # Then load quant F16
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 8K context, about 8.9–12 tok/s. Weights 12.8 GB plus 1.03 GB of KV cache — 14.2 GB of 14.7 GB usable.
llama-server -hf unsloth/gemma-3-27b-it-GGUF:UD-Q3_K_XL -c 8192 -ngl 62
ollama run gemma
lms get unsloth/gemma-3-27b-it-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q3_K_XL, 8K context, about 8.9–12 tok/s. Weights 12.8 GB plus 1.03 GB of KV cache — 14.2 GB of 14.7 GB usable.
llama-server -hf unsloth/medgemma-27b-text-it-GGUF:UD-Q3_K_XL -c 8192 -ngl 62
ollama run medgemma
lms get unsloth/medgemma-27b-text-it-GGUF # Then load quant UD-Q3_K_XL
Tight fit fully on your GeForce RTX 4060 Ti 16GB at Q3_K_S, 4K context, about 9.2–13 tok/s. Weights 12.3 GB plus 1.41 GB of KV cache — 14.1 GB of 14.7 GB usable.
llama-server -hf unsloth/gemma-4-31B-it-GGUF:Q3_K_S -c 4096 -ngl 60
ollama run gemma
lms get unsloth/gemma-4-31B-it-GGUF # Then load quant Q3_K_S
Tight fit fully on your GeForce RTX 4060 Ti 16GB at UD-Q8_K_XL, 8K context, about 9.4–13 tok/s. Weights 12.7 GB plus 0.81 GB of KV cache — 13.9 GB of 14.7 GB usable.
llama-server -hf unsloth/gemma-4-12b-it-GGUF:UD-Q8_K_XL -c 8192 -ngl 48
ollama run gemma
lms get unsloth/gemma-4-12b-it-GGUF # Then load quant UD-Q8_K_XL
Partial offload on your GeForce RTX 4060 Ti 16GB: Q4_K_S at 4K context, 27 of 80 layers on the GPU and 24.8 GB in system RAM. Expect about 1.6–2.1 tok/s.
llama-server -hf unsloth/Llama-3.3-70B-Instruct-GGUF:Q4_K_S -c 4096 -ngl 27
No Ollama library match for this repo.
lms get unsloth/Llama-3.3-70B-Instruct-GGUF # Then load quant Q4_K_S
Partial offload on your GeForce RTX 4060 Ti 16GB: Q4_K_S at 4K context, 27 of 80 layers on the GPU and 24.8 GB in system RAM. Expect about 1.6–2.1 tok/s.
llama-server -hf unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF:Q4_K_S -c 4096 -ngl 27
No Ollama library match for this repo.
lms get unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF # Then load quant Q4_K_S
Partial offload on your GeForce RTX 4060 Ti 16GB: Q4_K_S at 4K context, 27 of 80 layers on the GPU and 24.8 GB in system RAM. Expect about 1.6–2.1 tok/s.
llama-server -hf unsloth/Hermes-4-70B-GGUF:Q4_K_S -c 4096 -ngl 27
No Ollama library match for this repo.
lms get unsloth/Hermes-4-70B-GGUF # Then load quant Q4_K_S
Partial offload on your GeForce RTX 4060 Ti 16GB: IQ4_XS at 4K context, 28 of 80 layers on the GPU and 24.2 GB in system RAM. Expect about 1.6–2.2 tok/s.
llama-server -hf bartowski/Qwen2.5-72B-Instruct-GGUF:IQ4_XS -c 4096 -ngl 28
ollama run qwen2.5
lms get bartowski/Qwen2.5-72B-Instruct-GGUF # Then load quant IQ4_XS
Expert offload on your GeForce RTX 4060 Ti 16GB: UD-IQ4_XS keeps shared weights on the GPU and about 23.7 GB of experts in system RAM. About 22–30 tok/s. Active-parameter speed is approximate.
# Expert offload is approximate. llama.cpp can pin experts to CPU: # -ot "\\.ffn_.*_exps\\.=CPU" llama-server -hf unsloth/Qwen3-Coder-Next-GGUF:UD-IQ4_XS -c 8192 -ngl 99
ollama run qwen3-coder-next
lms get unsloth/Qwen3-Coder-Next-GGUF # Then load quant UD-IQ4_XS
Partial offload on your GeForce RTX 4060 Ti 16GB: UD-IQ2_XXS at 4K context, 34 of 88 layers on the GPU and 19.8 GB in system RAM. Expect about 1.9–2.6 tok/s.
llama-server -hf unsloth/Mistral-Medium-3.5-128B-GGUF:UD-IQ2_XXS -c 4096 -ngl 34
ollama run mistral
lms get unsloth/Mistral-Medium-3.5-128B-GGUF # Then load quant UD-IQ2_XXS
Partial offload on your GeForce RTX 4060 Ti 16GB: UD-IQ2_XXS at 4K context, 35 of 88 layers on the GPU and 19.3 GB in system RAM. Expect about 1.9–2.6 tok/s.
llama-server -hf unsloth/Devstral-2-123B-Instruct-2512-GGUF:UD-IQ2_XXS -c 4096 -ngl 35
ollama run devstral
lms get unsloth/Devstral-2-123B-Instruct-2512-GGUF # Then load quant UD-IQ2_XXS
Expert offload on your GeForce RTX 4060 Ti 16GB: UD-Q2_K_XL keeps shared weights on the GPU and about 23.9 GB of experts in system RAM. About 21–29 tok/s. Active-parameter speed is approximate.
# Expert offload is approximate. llama.cpp can pin experts to CPU: # -ot "\\.ffn_.*_exps\\.=CPU" llama-server -hf unsloth/Mistral-Small-4-119B-2603-GGUF:UD-Q2_K_XL -c 8192 -ngl 99
ollama run mistral
lms get unsloth/Mistral-Small-4-119B-2603-GGUF # Then load quant UD-Q2_K_XL
Partial offload on your GeForce RTX 4060 Ti 16GB: Q4_K_M at 4K context, 46 of 64 layers on the GPU and 5.25 GB in system RAM. Expect about 4.7–6.5 tok/s.
llama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-32B-GGUF:Q4_K_M -c 4096 -ngl 46
No Ollama library match for this repo.
lms get unsloth/DeepSeek-R1-Distill-Qwen-32B-GGUF # Then load quant Q4_K_M
Partial offload on your GeForce RTX 4060 Ti 16GB: UD-IQ2_XXS at 4K context, 18 of 48 layers on the GPU and 23.6 GB in system RAM. Expect about 20–27 tok/s.
llama-server -hf unsloth/Qwen3.5-122B-A10B-MTP-GGUF:UD-IQ2_XXS -c 4096 -ngl 18
ollama run qwen3.5
lms get unsloth/Qwen3.5-122B-A10B-MTP-GGUF # Then load quant UD-IQ2_XXS
Partial offload on your GeForce RTX 4060 Ti 16GB: UD-IQ1_M at 4K context, 16 of 46 layers on the GPU and 24.0 GB in system RAM. Expect about 7.4–10 tok/s.
llama-server -hf unsloth/GLM-4.5-Air-GGUF:UD-IQ1_M -c 4096 -ngl 16
No Ollama library match for this repo.
lms get unsloth/GLM-4.5-Air-GGUF # Then load quant UD-IQ1_M
Partial offload on your GeForce RTX 4060 Ti 16GB: UD-IQ2_XXS at 4K context, 18 of 48 layers on the GPU and 21.5 GB in system RAM. Expect about 8.1–11 tok/s.
llama-server -hf unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF:UD-IQ2_XXS -c 4096 -ngl 18
No Ollama library match for this repo.
lms get unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF # Then load quant UD-IQ2_XXS
Partial offload on your GeForce RTX 4060 Ti 16GB: UD-IQ2_XXS at 4K context, 16 of 46 layers on the GPU and 25.0 GB in system RAM. Expect about 7.1–9.7 tok/s.
llama-server -hf unsloth/GLM-4.6V-GGUF:UD-IQ2_XXS -c 4096 -ngl 16
No Ollama library match for this repo.
lms get unsloth/GLM-4.6V-GGUF # Then load quant UD-IQ2_XXS
32.9B · 3.0B active · nemotron_h_moe · 1024K max
Expert offload on your GeForce RTX 4060 Ti 16GB: UD-Q4_K_M keeps shared weights on the GPU and about 19.2 GB of experts in system RAM. About 58–79 tok/s. Active-parameter speed is approximate.
# Expert offload is approximate. llama.cpp can pin experts to CPU: # -ot "\\.ffn_.*_exps\\.=CPU" llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:UD-Q4_K_M -c 8192 -ngl 99
No Ollama library match for this repo.
lms get unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF # Then load quant UD-Q4_K_M
Expert offload on your GeForce RTX 4060 Ti 16GB: UD-Q2_K_XL keeps shared weights on the GPU and about 25.8 GB of experts in system RAM. About 82–111 tok/s. Active-parameter speed is approximate.
# Expert offload is approximate. llama.cpp can pin experts to CPU: # -ot "\\.ffn_.*_exps\\.=CPU" llama-server -hf unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF:UD-Q2_K_XL -c 8192 -ngl 99
ollama run qwen3
lms get unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF # Then load quant UD-Q2_K_XL
Expert offload on your GeForce RTX 4060 Ti 16GB: UD-Q2_K_XL keeps shared weights on the GPU and about 25.8 GB of experts in system RAM. About 82–111 tok/s. Active-parameter speed is approximate.
# Expert offload is approximate. llama.cpp can pin experts to CPU: # -ot "\\.ffn_.*_exps\\.=CPU" llama-server -hf unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF:UD-Q2_K_XL -c 8192 -ngl 99
ollama run qwen3
lms get unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF # Then load quant UD-Q2_K_XL
31.6B · 3.0B active · nemotron_h_moe · 1024K max
Expert offload on your GeForce RTX 4060 Ti 16GB: UD-Q4_K_M keeps shared weights on the GPU and about 18.0 GB of experts in system RAM. About 59–80 tok/s. Active-parameter speed is approximate.
# Expert offload is approximate. llama.cpp can pin experts to CPU: # -ot "\\.ffn_.*_exps\\.=CPU" llama-server -hf unsloth/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF:UD-Q4_K_M -c 8192 -ngl 99
No Ollama library match for this repo.
lms get unsloth/NVIDIA-Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF # Then load quant UD-Q4_K_M
Expert offload on your GeForce RTX 4060 Ti 16GB: Q4_K_M keeps shared weights on the GPU and about 18.5 GB of experts in system RAM. About 57–78 tok/s. Active-parameter speed is approximate.
# Expert offload is approximate. llama.cpp can pin experts to CPU: # -ot "\\.ffn_.*_exps\\.=CPU" llama-server -hf unsloth/Nemotron-3-Nano-30B-A3B-GGUF:Q4_K_M -c 8192 -ngl 99
ollama run nemotron
lms get unsloth/Nemotron-3-Nano-30B-A3B-GGUF # Then load quant Q4_K_M
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant Q2_K is 294.3 GB before KV cache, and usable memory is 14.7 GB.
# Quant size is estimated — the file may not be published yet llama-server -hf antirez/deepseek-v4.1-flash-gguf:Q2_K -c 2048 -ngl 0
ollama run deepseek-v4.1-flash
# Quant size is estimated — the file may not be published yet lms get antirez/deepseek-v4.1-flash-gguf # Then load quant Q2_K
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 201.8 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/GLM-5.3-GGUF:UD-IQ1_S -c 2048 -ngl 0
ollama run glm-5.3
lms get unsloth/GLM-5.3-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 130.6 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/DeepSeek-R1-GGUF:UD-IQ1_S -c 2048 -ngl 0
ollama run deepseek-r1
lms get unsloth/DeepSeek-R1-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 179.1 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/DeepSeek-V3.1-GGUF:UD-IQ1_S -c 2048 -ngl 0
ollama run deepseek-v3.1
lms get unsloth/DeepSeek-V3.1-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 171.5 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/DeepSeek-V3.2-GGUF:UD-IQ1_S -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/DeepSeek-V3.2-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_M is 139.4 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF:UD-IQ1_M -c 2048 -ngl 0
ollama run qwen3
lms get unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF # Then load quant UD-IQ1_M
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 112.5 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF:UD-IQ1_S -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_M is 99.5 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/Qwen3.5-397B-A17B-GGUF:UD-IQ1_M -c 2048 -ngl 0
ollama run qwen3.5
lms get unsloth/Qwen3.5-397B-A17B-GGUF # Then load quant UD-IQ1_M
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 90.3 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/GLM-4.6-GGUF:UD-IQ1_S -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/GLM-4.6-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 86.7 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/GLM-5.3-Flash-GGUF:UD-IQ1_S -c 2048 -ngl 0
ollama run glm-5.3-flash
lms get unsloth/GLM-5.3-Flash-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 81.2 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/MiMo-V2-Flash-GGUF:UD-IQ1_S -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/MiMo-V2-Flash-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 76.9 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/DeepSeek-V4-Flash-0731-GGUF:UD-IQ1_S -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/DeepSeek-V4-Flash-0731-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 76.8 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF:UD-IQ1_S -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-Q2_K_XL is 82.7 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF:UD-Q2_K_XL -c 2048 -ngl 0
ollama run qwen3
lms get unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF # Then load quant UD-Q2_K_XL
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-Q2_K_XL is 82.1 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF:UD-Q2_K_XL -c 2048 -ngl 0
ollama run qwen3
lms get unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF # Then load quant UD-Q2_K_XL
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 59.6 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/MiniMax-M2-GGUF:UD-IQ1_S -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/MiniMax-M2-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_M is 52.9 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/Step-3.7-Flash-GGUF:UD-IQ1_M -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/Step-3.7-Flash-GGUF # Then load quant UD-IQ1_M
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant Q3_K_S is 58.3 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/gpt-oss-120b-GGUF:Q3_K_S -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/gpt-oss-120b-GGUF # Then load quant Q3_K_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ1_S is 67.6 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ1_S -c 2048 -ngl 0
ollama run qwen3.8-flash-next
lms get unsloth/Qwen3.8-Flash-Next-GGUF # Then load quant UD-IQ1_S
Won't fit on GeForce RTX 4060 Ti 16GB. The smallest quant UD-IQ2_XXS is 49.1 GB before KV cache, and usable memory is 14.7 GB.
llama-server -hf unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF:UD-IQ2_XXS -c 2048 -ngl 0
No Ollama library match for this repo.
lms get unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF # Then load quant UD-IQ2_XXS
Or browse 16 GB, 24 GB, Apple silicon, and what arrived recently.