Which quantization should I use at this memory size?

The table shows the best quant that still leaves at least 2k tokens of context. Q4_K_M is the practical default; step up to Q5/Q6 only when it fits with room to spare.

Local AI checker

Best Local LLMs for 16GB VRAM (2026)

Q: Best Local LLMs for 16GB VRAM?

Top picks: Mistral Small 3.2 24B at IQ4_XS (12.76GB download); gpt-oss-20b at Q8_0 (12.11GB download); Qwen3 14B at Q6_K (12.12GB download); Qwen2.5 Coder 14B at Q6_K (12.12GB download).

RTX 4080/5080/4060 Ti 16GB, RX 7800 XT/9070, Arc A770 · ~15 GB usable · real GGUF sizes, refreshed automatically

16GB comfortably runs 14B at high quants and reaches into the 20-24B class at Q4 — including MoE tricks like gpt-oss-20b that run faster than their size suggests.

Best overall

Mistral Small 3.2 24B IQ4_XS

European all-rounder with vision; great on a 24GB card, usable at Q4 on 16GB.

Best for coding

Mistral Small 3.2 24B IQ4_XS

European all-rounder with vision; great on a 24GB card, usable at Q4 on 16GB.

Best for reasoning

gpt-oss-20b Q8_0

OpenAI's open-weight MoE — only ~3.6B active params, so it runs faster than its size suggests.

With vision

Mistral Small 3.2 24B IQ4_XS

European all-rounder with vision; great on a 24GB card, usable at Q4 on 16GB.

Model	Params	Best quant	Download	Context	Run it
Mistral Small 3.2 24Bchatcoding	24B	IQ4_XS	12.76 GB	~4k	`ollama run mistral-small3.2`
gpt-oss-20bchatreasoning	21B	Q8_0	12.11 GB	~4k	`ollama run gpt-oss:20b`
Qwen3 14Bchatcoding	14.8B	Q6_K	12.12 GB	~8k	`ollama run qwen3:14b`
Qwen2.5 Coder 14Bcoding	14.8B	Q6_K	12.12 GB	~8k	`ollama run qwen2.5-coder:14b`
DeepSeek R1 Distill 14Breasoning	14.8B	Q6_K	12.12 GB	~8k	`ollama run deepseek-r1:14b`
Phi-4 14Breasoningcoding	14.7B	Q6_K	12.03 GB	~8k	`ollama run phi4:14b`
Mistral Nemo 12Bchatlong-context	12.2B	Q8_0	13.02 GB	~4k	`ollama run mistral-nemo:12b`
Gemma 3 12Bchatvision	12.2B	Q8_0	12.51 GB	~8k	`ollama run gemma3:12b`
Qwen2.5 VL 7Bvisionchat	8.3B	Q8_0	8.95 GB	~32k	`ollama run qwen2.5vl:7b`
Qwen3 8Bchatcoding	8.2B	Q8_0	8.71 GB	~32k	`ollama run qwen3:8b`
Llama 3.1 8Bchat	8B	Q8_0	8.54 GB	~32k	`ollama run llama3.1:8b`
Qwen2.5 Coder 7Bcoding	7.6B	Q8_0	8.1 GB	~32k	`ollama run qwen2.5-coder:7b`
DeepSeek R1 Distill 7Breasoning	7.6B	Q8_0	8.1 GB	~32k	`ollama run deepseek-r1:7b`
Gemma 3 4Bchatvision	4.3B	Q8_0	4.13 GB	~128k	`ollama run gemma3:4b`
Qwen3 4Bchatreasoning	4B	Q8_0	4.28 GB	~128k	`ollama run qwen3:4b`
Llama 3.2 3Bchat	3.2B	Q8_0	3.42 GB	~128k	`ollama run llama3.2:3b`
Qwen3 1.7Bchat	1.7B	Q8_0	2.17 GB	~128k	`ollama run qwen3:1.7b`
Llama 3.2 1Bchat	1.2B	Q8_0	1.32 GB	~128k	`ollama run llama3.2:1b`

Out of reach at this size: Gemma 3 27B, Qwen3 30B A3B (MoE), Qwen2.5 Coder 32B, Qwen3 32B, DeepSeek R1 Distill 32B, Llama 3.3 70B, DeepSeek R1 Distill 70B, Llama 4 Scout (109B MoE), gpt-oss-120b. For those, the cheapest capable API right now is gpt-oss-120b — compare on the calculator.

Check your exact GPU/Mac 8GBGB 12GBGB 24GBGB 32GBGB 48GBGB Mac 16gb Mac 32gb Mac 64gb

Estimates: file size × 1.05 + KV-cache + 0.4GB buffer; GPU usable = VRAM − 0.6GB, Apple usable = 72% of unified memory. File sizes are real GGUF bytes from Hugging Face, refreshed automatically. No affiliate links.