Which open models fit on 16, 24, 32, 48 and 96 GB GPUs

Estimated VRAM for popular open models at Q4 and Q8, 8k and 32k context, with the smallest GPU that fits each one. From public architecture numbers.

Updated 8 September 2026 · 6 min read · estimates are labelled as estimates

Estimated VRAM for popular open models, and the smallest memory tier each fits with about 5% headroom. Everything is computed from public architecture numbers with the formula in the calculator; these are estimates to plan with, not measurements to bank on.

The table

ModelParamsQ4 · 8kfitsQ4 · 32kfitsQ8 · 8kfits
Llama 3.2 3B 3.2B 3.4 GB8 GB 6.2 GB8 GB 5 GB8 GB
Llama 3.1 8B 8B 6.4 GB8 GB 9.6 GB12 GB 10.4 GB12 GB
Qwen3 8B 8.2B 6.7 GB8 GB 10.3 GB12 GB 10.7 GB12 GB
Gemma 3 12B 12.2B 11.1 GB12 GB 20.7 GB24 GB 17.2 GB24 GB
Qwen3 14B 14.8B 10.8 GB12 GB 14.8 GB16 GB 18.2 GB24 GB
Phi-4 14B 14.7B 11 GB12 GB 16.1 GB24 GB 18.4 GB24 GB
gpt-oss 20B · 3.6B active 21B 13.6 GB16 GB 14.8 GB16 GB 24.1 GB32 GB
Mistral Small 3.1 24B 24B 16.3 GB24 GB 20.3 GB24 GB 28.3 GB32 GB
Gemma 3 27B 27.4B 21.2 GB24 GB 33.7 GB48 GB 34.9 GB48 GB
Qwen3 30B-A3B (MoE) · 3.3B active 30.5B 19.7 GB24 GB 22.1 GB24 GB 34.9 GB48 GB
Qwen3 32B 32.8B 22.4 GB24 GB 28.9 GB32 GB 38.8 GB48 GB
Qwen2.5 Coder 32B 32.8B 22.4 GB24 GB 28.9 GB32 GB 38.8 GB48 GB
DeepSeek-R1 Distill Qwen 32B 32.8B 22.4 GB24 GB 28.9 GB32 GB 38.8 GB48 GB
Llama 3.3 70B 70.6B 45.8 GB80 GB 53.8 GB80 GB 81 GB96 GB
DeepSeek-R1 Distill Llama 70B 70.6B 45.8 GB80 GB 53.8 GB80 GB 81 GB96 GB
gpt-oss 120B · 5.1B active 117B 71.7 GB80 GB 73.5 GB80 GB 130.1 GBmulti-GPU

How to read it

  • Q4 · 8k is the everyday setting: Q4_K_M weights, 8,192 tokens of context. Most chat and coding sessions live here.
  • Q4 · 32k shows what long context costs. The difference between the two columns is almost entirely KV cache.
  • Q8 · 8k is near-lossless quality. If it fits, prefer it for tasks where precision shows, like long reasoning and code.
  • Models marked with an active-parameter count are mixture-of-experts: all experts load into memory, so the weights are large, but only a few billion parameters compute per token, which makes them fast.
  • Gemma 3 uses sliding-window attention on most layers, so its real KV cache is smaller than the estimate. gpt-oss ships in MXFP4, close to the Q4 row.

Each model name links to its own page with memory at three context lengths and speed estimates per GPU. To see a model that is not here, or a different context length, use the calculator with custom parameters.

This table asks which card a model needs. The opposite question — which models a card you already own can run, with how much context left over — is answered per card on the GPU pages.

Method

Weights = parameters × bytes per parameter (Q4_K_M 0.58, Q8_0 1.06, effective averages including scales and unquantised embedding and output layers). KV cache = 2 × layers × KV heads × head dimension × context × 2 bytes. Overhead = 0.5 GB + 4% of weights. A tier "fits" when the total is at or under 95% of its memory. Architecture numbers come from each model's public config.json or model card:

  • Llama 3.2 3B: meta-llama/Llama-3.2-3B config.json
  • Llama 3.1 8B: meta-llama/Llama-3.1-8B config.json
  • Qwen3 8B: Qwen/Qwen3-8B config.json
  • Gemma 3 12B: google/gemma-3-12b-it config.json
  • Qwen3 14B: Qwen/Qwen3-14B config.json
  • Phi-4 14B: microsoft/phi-4 config.json
  • gpt-oss 20B: openai/gpt-oss-20b model card
  • Mistral Small 3.1 24B: mistralai/Mistral-Small-3.1-24B config.json
  • Gemma 3 27B: google/gemma-3-27b-it config.json
  • Qwen3 30B-A3B (MoE): Qwen/Qwen3-30B-A3B config.json
  • Qwen3 32B: Qwen/Qwen3-32B config.json
  • Qwen2.5 Coder 32B: Qwen/Qwen2.5-Coder-32B-Instruct config.json
  • DeepSeek-R1 Distill Qwen 32B: deepseek-ai/DeepSeek-R1-Distill-Qwen-32B config.json
  • Llama 3.3 70B: meta-llama/Llama-3.3-70B-Instruct config.json
  • DeepSeek-R1 Distill Llama 70B: deepseek-ai/DeepSeek-R1-Distill-Llama-70B config.json
  • gpt-oss 120B: openai/gpt-oss-120b model card