Which quantization should I use at this memory size?

The table shows the best quant that still leaves at least 2k tokens of context. Q4_K_M is the practical default; step up to Q5/Q6 only when it fits with room to spare.

Local AI checker

Best Local LLMs for 8GB VRAM (2026)

Q: Best Local LLMs for 8GB VRAM?

Top picks: Qwen2.5 VL 7B at Q4_K_M (4.68GB download); Qwen3 8B at Q5_K_M (5.85GB download); Llama 3.1 8B at Q5_K_M (5.73GB download); Qwen2.5 Coder 7B at Q6_K (6.25GB download).

RTX 4060/3070/5060, RX 7600, most gaming laptops · ~7 GB usable · real GGUF sizes, refreshed automatically

8GB is the most common gaming-card size and the entry ticket to local AI. The sweet spot here is 7-8B models at Q4/Q5 — modern ones are genuinely useful for chat, drafting and light coding.

Best overall

Qwen2.5 VL 7B Q4_K_M

Strong local vision-language model: screenshots, documents, OCR-ish tasks.

Best for coding

Qwen3 8B Q5_K_M

Best all-rounder at 8B right now; optional thinking mode.

Best for reasoning

Qwen3 8B Q5_K_M

Best all-rounder at 8B right now; optional thinking mode.

With vision

Qwen2.5 VL 7B Q4_K_M

Strong local vision-language model: screenshots, documents, OCR-ish tasks.

Model	Params	Best quant	Download	Context	Run it
Qwen2.5 VL 7Bvisionchat	8.3B	Q4_K_M	4.68 GB	~16k	`ollama run qwen2.5vl:7b`
Qwen3 8Bchatcoding	8.2B	Q5_K_M	5.85 GB	~4k	`ollama run qwen3:8b`
Llama 3.1 8Bchat	8B	Q5_K_M	5.73 GB	~8k	`ollama run llama3.1:8b`
Qwen2.5 Coder 7Bcoding	7.6B	Q6_K	6.25 GB	~4k	`ollama run qwen2.5-coder:7b`
DeepSeek R1 Distill 7Breasoning	7.6B	Q6_K	6.25 GB	~4k	`ollama run deepseek-r1:7b`
Gemma 3 4Bchatvision	4.3B	Q8_0	4.13 GB	~32k	`ollama run gemma3:4b`
Qwen3 4Bchatreasoning	4B	Q8_0	4.28 GB	~32k	`ollama run qwen3:4b`
Llama 3.2 3Bchat	3.2B	Q8_0	3.42 GB	~64k	`ollama run llama3.2:3b`
Qwen3 1.7Bchat	1.7B	Q8_0	2.17 GB	~128k	`ollama run qwen3:1.7b`
Llama 3.2 1Bchat	1.2B	Q8_0	1.32 GB	~128k	`ollama run llama3.2:1b`

Out of reach at this size: Mistral Nemo 12B, Gemma 3 12B, Phi-4 14B, Qwen3 14B, Qwen2.5 Coder 14B, DeepSeek R1 Distill 14B, gpt-oss-20b, Mistral Small 3.2 24B, Gemma 3 27B, Qwen3 30B A3B (MoE), Qwen2.5 Coder 32B, Qwen3 32B, DeepSeek R1 Distill 32B, Llama 3.3 70B, DeepSeek R1 Distill 70B, Llama 4 Scout (109B MoE), gpt-oss-120b. For those, the cheapest capable API right now is gpt-oss-120b — compare on the calculator.

Check your exact GPU/Mac 12GBGB 16GBGB 24GBGB 32GBGB 48GBGB Mac 16gb Mac 32gb Mac 64gb

Estimates: file size × 1.05 + KV-cache + 0.4GB buffer; GPU usable = VRAM − 0.6GB, Apple usable = 72% of unified memory. File sizes are real GGUF bytes from Hugging Face, refreshed automatically. No affiliate links.