Which quantization should I use at this memory size?

The table shows the best quant that still leaves at least 2k tokens of context. Q4_K_M is the practical default; step up to Q5/Q6 only when it fits with room to spare.

Local AI checker

Best Local LLMs for 24GB VRAM (2026)

Q: Best Local LLMs for 24GB VRAM?

Top picks: Qwen2.5 Coder 32B at Q4_K_M (19.85GB download); Qwen3 32B at Q4_K_M (19.76GB download); DeepSeek R1 Distill 32B at Q4_K_M (19.85GB download); Qwen3 30B A3B (MoE) at Q4_K_M (18.63GB download).

RTX 4090/3090, RX 7900 XTX · ~23 GB usable · real GGUF sizes, refreshed automatically

The enthusiast tier: 27-32B dense models fit at Q4/Q5, which is where local models start seriously competing with mid-tier hosted APIs for coding and reasoning.

Best overall

Qwen2.5 Coder 32B Q4_K_M

The strongest dedicated local coder that fits a 24GB card.

Best for coding

Qwen2.5 Coder 32B Q4_K_M

The strongest dedicated local coder that fits a 24GB card.

Best for reasoning

Qwen3 32B Q4_K_M

Dense 32B flagship; the serious pick for 24GB cards.

With vision

Gemma 3 27B Q5_K_M

Top open chat quality per GB in the Gemma line; vision included.

Model	Params	Best quant	Download	Context	Run it
Qwen2.5 Coder 32Bcoding	32.8B	Q4_K_M	19.85 GB	~4k	`ollama run qwen2.5-coder:32b`
Qwen3 32Bchatcoding	32.8B	Q4_K_M	19.76 GB	~4k	`ollama run qwen3:32b`
DeepSeek R1 Distill 32Breasoning	32.8B	Q4_K_M	19.85 GB	~4k	`ollama run deepseek-r1:32b`
Qwen3 30B A3B (MoE)chatcoding	30.5B	Q4_K_M	18.63 GB	~4k	`ollama run qwen3:30b`
Gemma 3 27Bchatvision	27B	Q5_K_M	19.27 GB	~4k	`ollama run gemma3:27b`
Mistral Small 3.2 24Bchatcoding	24B	Q6_K	19.35 GB	~4k	`ollama run mistral-small3.2`
gpt-oss-20bchatreasoning	21B	Q8_0	12.11 GB	~32k	`ollama run gpt-oss:20b`
Qwen3 14Bchatcoding	14.8B	Q8_0	15.7 GB	~16k	`ollama run qwen3:14b`
Qwen2.5 Coder 14Bcoding	14.8B	Q8_0	15.7 GB	~16k	`ollama run qwen2.5-coder:14b`
DeepSeek R1 Distill 14Breasoning	14.8B	Q8_0	15.7 GB	~16k	`ollama run deepseek-r1:14b`
Phi-4 14Breasoningcoding	14.7B	Q8_0	15.58 GB	~16k	`ollama run phi4:14b`
Mistral Nemo 12Bchatlong-context	12.2B	Q8_0	13.02 GB	~32k	`ollama run mistral-nemo:12b`
Gemma 3 12Bchatvision	12.2B	Q8_0	12.51 GB	~32k	`ollama run gemma3:12b`
Qwen2.5 VL 7Bvisionchat	8.3B	Q8_0	8.95 GB	~64k	`ollama run qwen2.5vl:7b`
Qwen3 8Bchatcoding	8.2B	Q8_0	8.71 GB	~64k	`ollama run qwen3:8b`
Llama 3.1 8Bchat	8B	Q8_0	8.54 GB	~64k	`ollama run llama3.1:8b`
Qwen2.5 Coder 7Bcoding	7.6B	Q8_0	8.1 GB	~128k	`ollama run qwen2.5-coder:7b`
DeepSeek R1 Distill 7Breasoning	7.6B	Q8_0	8.1 GB	~128k	`ollama run deepseek-r1:7b`
Gemma 3 4Bchatvision	4.3B	Q8_0	4.13 GB	~128k	`ollama run gemma3:4b`
Qwen3 4Bchatreasoning	4B	Q8_0	4.28 GB	~128k	`ollama run qwen3:4b`
Llama 3.2 3Bchat	3.2B	Q8_0	3.42 GB	~128k	`ollama run llama3.2:3b`
Qwen3 1.7Bchat	1.7B	Q8_0	2.17 GB	~128k	`ollama run qwen3:1.7b`
Llama 3.2 1Bchat	1.2B	Q8_0	1.32 GB	~128k	`ollama run llama3.2:1b`

Out of reach at this size: Llama 3.3 70B, DeepSeek R1 Distill 70B, Llama 4 Scout (109B MoE), gpt-oss-120b. For those, the cheapest capable API right now is gpt-oss-120b — compare on the calculator.

Check your exact GPU/Mac 8GBGB 12GBGB 16GBGB 32GBGB 48GBGB Mac 16gb Mac 32gb Mac 64gb

Estimates: file size × 1.05 + KV-cache + 0.4GB buffer; GPU usable = VRAM − 0.6GB, Apple usable = 72% of unified memory. File sizes are real GGUF bytes from Hugging Face, refreshed automatically. No affiliate links.