Which quantization should I use at this memory size?

The table shows the best quant that still leaves at least 2k tokens of context. Q4_K_M is the practical default; step up to Q5/Q6 only when it fits with room to spare.

Local AI checker

Best Local LLMs for 32GB VRAM (2026)

Q: Best Local LLMs for 32GB VRAM?

Top picks: Qwen2.5 Coder 32B at Q6_K (26.89GB download); Qwen3 32B at Q6_K (26.88GB download); DeepSeek R1 Distill 32B at Q6_K (26.89GB download); Qwen3 30B A3B (MoE) at Q6_K (25.1GB download).

RTX 5090, dual 16GB cards · ~31 GB usable · real GGUF sizes, refreshed automatically

32GB runs the whole 32B class at comfortable quants with long context, and MoE 30B models with room to spare. 70B is still out of reach on a single card — that needs the next tier.

Best overall

Qwen2.5 Coder 32B Q6_K

The strongest dedicated local coder that fits a 24GB card.

Best for coding

Qwen2.5 Coder 32B Q6_K

The strongest dedicated local coder that fits a 24GB card.

Best for reasoning

Qwen3 32B Q6_K

Dense 32B flagship; the serious pick for 24GB cards.

With vision

Gemma 3 27B Q8_0

Top open chat quality per GB in the Gemma line; vision included.

Model	Params	Best quant	Download	Context	Run it
Qwen2.5 Coder 32Bcoding	32.8B	Q6_K	26.89 GB	~4k	`ollama run qwen2.5-coder:32b`
Qwen3 32Bchatcoding	32.8B	Q6_K	26.88 GB	~4k	`ollama run qwen3:32b`
DeepSeek R1 Distill 32Breasoning	32.8B	Q6_K	26.89 GB	~4k	`ollama run deepseek-r1:32b`
Qwen3 30B A3B (MoE)chatcoding	30.5B	Q6_K	25.1 GB	~8k	`ollama run qwen3:30b`
Gemma 3 27Bchatvision	27B	Q8_0	28.71 GB	~2k	`ollama run gemma3:27b`
Mistral Small 3.2 24Bchatcoding	24B	Q8_0	25.05 GB	~8k	`ollama run mistral-small3.2`
gpt-oss-20bchatreasoning	21B	Q8_0	12.11 GB	~32k	`ollama run gpt-oss:20b`
Qwen3 14Bchatcoding	14.8B	Q8_0	15.7 GB	~64k	`ollama run qwen3:14b`
Qwen2.5 Coder 14Bcoding	14.8B	Q8_0	15.7 GB	~64k	`ollama run qwen2.5-coder:14b`
DeepSeek R1 Distill 14Breasoning	14.8B	Q8_0	15.7 GB	~64k	`ollama run deepseek-r1:14b`
Phi-4 14Breasoningcoding	14.7B	Q8_0	15.58 GB	~64k	`ollama run phi4:14b`
Mistral Nemo 12Bchatlong-context	12.2B	Q8_0	13.02 GB	~64k	`ollama run mistral-nemo:12b`
Gemma 3 12Bchatvision	12.2B	Q8_0	12.51 GB	~64k	`ollama run gemma3:12b`
Qwen2.5 VL 7Bvisionchat	8.3B	Q8_0	8.95 GB	~128k	`ollama run qwen2.5vl:7b`
Qwen3 8Bchatcoding	8.2B	Q8_0	8.71 GB	~128k	`ollama run qwen3:8b`
Llama 3.1 8Bchat	8B	Q8_0	8.54 GB	~128k	`ollama run llama3.1:8b`
Qwen2.5 Coder 7Bcoding	7.6B	Q8_0	8.1 GB	~128k	`ollama run qwen2.5-coder:7b`
DeepSeek R1 Distill 7Breasoning	7.6B	Q8_0	8.1 GB	~128k	`ollama run deepseek-r1:7b`
Gemma 3 4Bchatvision	4.3B	Q8_0	4.13 GB	~128k	`ollama run gemma3:4b`
Qwen3 4Bchatreasoning	4B	Q8_0	4.28 GB	~128k	`ollama run qwen3:4b`
Llama 3.2 3Bchat	3.2B	Q8_0	3.42 GB	~128k	`ollama run llama3.2:3b`
Qwen3 1.7Bchat	1.7B	Q8_0	2.17 GB	~128k	`ollama run qwen3:1.7b`
Llama 3.2 1Bchat	1.2B	Q8_0	1.32 GB	~128k	`ollama run llama3.2:1b`

Out of reach at this size: Llama 3.3 70B, DeepSeek R1 Distill 70B, Llama 4 Scout (109B MoE), gpt-oss-120b. For those, the cheapest capable API right now is gpt-oss-120b — compare on the calculator.

Check your exact GPU/Mac 8GBGB 12GBGB 16GBGB 24GBGB 48GBGB Mac 16gb Mac 32gb Mac 64gb

Estimates: file size × 1.05 + KV-cache + 0.4GB buffer; GPU usable = VRAM − 0.6GB, Apple usable = 72% of unified memory. File sizes are real GGUF bytes from Hugging Face, refreshed automatically. No affiliate links.