Which quantization should I use at this memory size?

The table shows the best quant that still leaves at least 2k tokens of context. Q4_K_M is the practical default; step up to Q5/Q6 only when it fits with room to spare.

Local AI checker

Best Local LLMs for 12GB VRAM (2026)

Q: Best Local LLMs for 12GB VRAM?

Top picks: Qwen3 14B at Q4_K_M (9GB download); Qwen2.5 Coder 14B at Q4_K_M (8.99GB download); DeepSeek R1 Distill 14B at Q4_K_M (8.99GB download); Phi-4 14B at Q4_K_M (9.05GB download).

RTX 3060 12GB / 4070 / 5070, RX 7700 XT, Arc B580 · ~11 GB usable · real GGUF sizes, refreshed automatically

12GB unlocks the 12-14B class at Q4/Q5 — a visible quality step over 8B, especially for coding and reasoning. The RTX 3060 12GB remains the budget king of local AI for a reason.

Best overall

Qwen3 14B Q4_K_M

The 12-16GB sweet spot: near-32B quality at half the memory.

Best for coding

Qwen3 14B Q4_K_M

The 12-16GB sweet spot: near-32B quality at half the memory.

Best for reasoning

Qwen3 14B Q4_K_M

The 12-16GB sweet spot: near-32B quality at half the memory.

With vision

Gemma 3 12B Q6_K

The 12GB-card default: strong chat + vision at a friendly size.

Model	Params	Best quant	Download	Context	Run it
Qwen3 14Bchatcoding	14.8B	Q4_K_M	9 GB	~4k	`ollama run qwen3:14b`
Qwen2.5 Coder 14Bcoding	14.8B	Q4_K_M	8.99 GB	~4k	`ollama run qwen2.5-coder:14b`
DeepSeek R1 Distill 14Breasoning	14.8B	Q4_K_M	8.99 GB	~4k	`ollama run deepseek-r1:14b`
Phi-4 14Breasoningcoding	14.7B	Q4_K_M	9.05 GB	~4k	`ollama run phi4:14b`
Mistral Nemo 12Bchatlong-context	12.2B	Q6_K	10.06 GB	~2k	`ollama run mistral-nemo:12b`
Gemma 3 12Bchatvision	12.2B	Q6_K	9.66 GB	~4k	`ollama run gemma3:12b`
Qwen2.5 VL 7Bvisionchat	8.3B	Q8_0	8.95 GB	~8k	`ollama run qwen2.5vl:7b`
Qwen3 8Bchatcoding	8.2B	Q8_0	8.71 GB	~8k	`ollama run qwen3:8b`
Llama 3.1 8Bchat	8B	Q8_0	8.54 GB	~16k	`ollama run llama3.1:8b`
Qwen2.5 Coder 7Bcoding	7.6B	Q8_0	8.1 GB	~16k	`ollama run qwen2.5-coder:7b`
DeepSeek R1 Distill 7Breasoning	7.6B	Q8_0	8.1 GB	~16k	`ollama run deepseek-r1:7b`
Gemma 3 4Bchatvision	4.3B	Q8_0	4.13 GB	~64k	`ollama run gemma3:4b`
Qwen3 4Bchatreasoning	4B	Q8_0	4.28 GB	~64k	`ollama run qwen3:4b`
Llama 3.2 3Bchat	3.2B	Q8_0	3.42 GB	~128k	`ollama run llama3.2:3b`
Qwen3 1.7Bchat	1.7B	Q8_0	2.17 GB	~128k	`ollama run qwen3:1.7b`
Llama 3.2 1Bchat	1.2B	Q8_0	1.32 GB	~128k	`ollama run llama3.2:1b`

Out of reach at this size: gpt-oss-20b, Mistral Small 3.2 24B, Gemma 3 27B, Qwen3 30B A3B (MoE), Qwen2.5 Coder 32B, Qwen3 32B, DeepSeek R1 Distill 32B, Llama 3.3 70B, DeepSeek R1 Distill 70B, Llama 4 Scout (109B MoE), gpt-oss-120b. For those, the cheapest capable API right now is gpt-oss-120b — compare on the calculator.

Check your exact GPU/Mac 8GBGB 16GBGB 24GBGB 32GBGB 48GBGB Mac 16gb Mac 32gb Mac 64gb

Estimates: file size × 1.05 + KV-cache + 0.4GB buffer; GPU usable = VRAM − 0.6GB, Apple usable = 72% of unified memory. File sizes are real GGUF bytes from Hugging Face, refreshed automatically. No affiliate links.