What LLM can I run on my GPU?

As a rule of thumb: 8GB VRAM runs 7-8B models at Q4, 12GB runs 12-14B, 16GB runs 14-24B, 24GB runs 27-32B, and 48GB+ is 70B territory. Pick your exact GPU in the checker above for real file sizes and context headroom.

Which quantization should I use?

Q4_K_M is the practical default — about 4.7 bits per weight with minimal quality loss. Use a bigger quant (Q5, Q6, Q8) only when the model still fits with context to spare; quality gains past Q6 are small.

Is running locally cheaper than the API?

Only at high volume. Electricity aside, a local model is free per token, but API prices for capable small models are now so low that light users rarely save money by buying hardware. Run the numbers on our pricing calculator before buying a GPU for AI.

What LLM can your machine actually run?

Pick your GPU or Mac. We match it against real GGUF file sizes from Hugging Face — not estimates — and tell you which models fit, at which quantization, with how much context, and the exact command to start. If a model is out of reach, we show what the hosted API costs instead.

27 models tracked · sizes refreshed 8 Jul 2026 · no affiliate links

1 · Your hardware

RTX 4060 (8GB) → usable for models: 7.4 GB (VRAM minus ~0.6GB display reserve)

2 · What runs on it

Qwen2.5 VL 7B

8.3Bvisionchat

comfortable fit

Strong local vision-language model: screenshots, documents, OCR-ish tasks.

best quant: Q4_K_Mdownload: 4.68 GBcontext headroom: ~16k tokens

or GGUF for LM Studio

Qwen3 8B

8.2Bchatcodingreasoning

tight fit

Best all-rounder at 8B right now; optional thinking mode.

best quant: Q5_K_Mdownload: 5.85 GBcontext headroom: ~4k tokens

or GGUF for LM Studio

Llama 3.1 8B

8Bchat

comfortable fit

The classic 8B baseline; huge fine-tune ecosystem.

best quant: Q5_K_Mdownload: 5.73 GBcontext headroom: ~8k tokens

or GGUF for LM Studio

Qwen2.5 Coder 7B

7.6Bcoding

tight fit

Purpose-built code model; pairs well with local IDE assistants.

best quant: Q6_Kdownload: 6.25 GBcontext headroom: ~4k tokens

or GGUF for LM Studio

DeepSeek R1 Distill 7B

7.6Breasoning

tight fit

Reasoning distill — shows chain-of-thought locally on modest hardware.

best quant: Q6_Kdownload: 6.25 GBcontext headroom: ~4k tokens

or GGUF for LM Studio

Gemma 3 4B

4.3Bchatvision

comfortable fit

Great small chat model with vision; the default pick for 6GB cards.

best quant: Q8_0download: 4.13 GBcontext headroom: ~32k tokens

or GGUF for LM Studiohosted API: $0.05/$0.1 per 1M

Qwen3 4B

4Bchatreasoning

comfortable fit

Thinking mode in a 4B — surprisingly capable for the size.

best quant: Q8_0download: 4.28 GBcontext headroom: ~32k tokens

or GGUF for LM Studio

Llama 3.2 3B

3.2Bchat

comfortable fit

Solid all-round small model for 4-6GB VRAM machines.

best quant: Q8_0download: 3.42 GBcontext headroom: ~64k tokens

or GGUF for LM Studio

Qwen3 1.7B

1.7Bchat

comfortable fit

Small but modern; better instruction-following than older 3B models.

best quant: Q8_0download: 2.17 GBcontext headroom: ~128k tokens

or GGUF for LM Studio

Llama 3.2 1B

1.2Bchat

comfortable fit

Fastest sane chat model; fine for autocomplete-style helpers on old laptops.

best quant: Q8_0download: 1.32 GBcontext headroom: ~128k tokens

or GGUF for LM Studio

Too big for this machine

These need more memory than you have — but every one of them (or a close equivalent) is an API call away, and at low volumes the API is usually cheaper than a GPU upgrade.

Mistral Nemo 12Bneeds ~9 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
Gemma 3 12Bneeds ~8 GB+→ via API: $0.05/$0.15 per 1M tokens
Phi-4 14Bneeds ~10 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
Qwen3 14Bneeds ~10 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
Qwen2.5 Coder 14Bneeds ~10 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
DeepSeek R1 Distill 14Bneeds ~10 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
gpt-oss-20bneeds ~14 GB+→ via API: $0.029/$0.14 per 1M tokens
Mistral Small 3.2 24Bneeds ~15 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
Gemma 3 27Bneeds ~17 GB+→ via API: $0.08/$0.16 per 1M tokens
Qwen3 30B A3B (MoE)needs ~19 GB+→ via API: $0.0482/$0.193 per 1M tokens
Qwen2.5 Coder 32Bneeds ~20 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
Qwen3 32Bneeds ~20 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
DeepSeek R1 Distill 32Bneeds ~20 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
Llama 3.3 70Bneeds ~41 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
DeepSeek R1 Distill 70Bneeds ~41 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
Llama 4 Scout (109B MoE)needs ~62 GB+→ cheapest capable API alternative: gpt-oss-120b at $0.06/1M blended
gpt-oss-120bneeds ~67 GB+→ via API: $0.03/$0.15 per 1M tokens

Compare the full API market on the pricing calculator and the value leaderboard.

Or browse by hardware tier

8GB VRAM 12GB VRAM 16GB VRAM 24GB VRAM 32GB VRAM 48GB+ VRAM Mac 16GB Mac 32GB Mac 64GB+

How the estimate works

Memory needed = real GGUF file size × 1.05 + KV-cache + 0.4GB runtime buffer. KV-cache is estimated at ~0.014GB per 1k tokens per billion parameters (a GQA-era heuristic; MoE models slightly overestimate, which errs on the safe side). Usable memory = VRAM − 0.6GB on dedicated GPUs, or 72% of unified memory on Apple Silicon (the default macOS GPU limit).

File sizes come straight from each model's GGUF repository on Hugging Face and are refreshed automatically. Tokens-per-second depends on memory bandwidth and backend, so we deliberately don't fake speed numbers — if it fits with headroom, it runs. New notable open models are added as they appear on our radar.