Which open models fit on 16, 24, 32, 48 and 96 GB GPUs

Estimated VRAM for popular open models, and the smallest memory tier each fits with about 5% headroom. Everything is computed from public architecture numbers with the formula in the calculator; these are estimates to plan with, not measurements to bank on.

The table

How to read it

• Q4 · 8k is the everyday setting: Q4_K_M weights, 8,192 tokens of context. Most chat and coding sessions live here.
• Q4 · 32k shows what long context costs. The difference between the two columns is almost entirely KV cache.
• Q8 · 8k is near-lossless quality. If it fits, prefer it for tasks where precision shows, like long reasoning and code.
• Models marked with an active-parameter count are mixture-of-experts: all experts load into memory, so the weights are large, but only a few billion parameters compute per token, which makes them fast.
• gpt-oss ships in MXFP4, close to the Q4 row.
• 16 of the 34 models here were published in 2026. Older models stay in the table because people still run them, and each older page links to its successor.

Each model name links to its own page with memory at three context lengths and speed estimates per GPU. To see a model that is not here, or a different context length, use the calculator with custom parameters.

This table asks which card a model needs. The opposite question (which models a card you already own can run, with how much context left over) is answered per card on the GPU pages.

Why newer models pay less for context

Compare the two Q4 columns and a pattern appears. Qwen3 32B, a standard transformer from 2025, goes from 22.4 GB to 28.9 GB when the context grows from 8k to 32k. Qwen3.8 27B, its 2026 successor, goes from 18 GB to 19.6 GB. At 32k tokens the first spends 8.6 GB on context and the second 2.3 GB. On a 24 GB card that is the difference between a model that runs out of room mid-document and one that does not.

Three designs are responsible, and a VRAM figure that ignores them is wrong by gigabytes:

• Hybrid attention (Qwen3.5 and later, Nemotron 3.5). Three layers in four are linear attention or Mamba, which keep a small fixed state instead of a cache. Only the remaining full-attention layers grow with the conversation.
• Sliding windows (Gemma 3 and 4, gpt-oss). Most layers only remember the last 128 to 1,024 tokens. A handful of global layers carry the long context: Gemma 4 31B spends 3.5 GB on a 32k context where a conventional cache of the same shape would be several times that.
• Latent attention (Mistral Small 4, GLM-4.7 Flash). Each layer caches one compressed vector per token rather than keys and values for every head. Mistral Small 4 is a 119B model whose 32k context costs 0.8 GB.

The catch is the runtime. These savings exist only when the software implements the matching cache. Current llama.cpp and vLLM builds do for the families above; an older build, or a less common runtime, may quietly store every layer at full length, and then the old arithmetic applies again. If a model uses far more memory than this table says at long context, that is the first thing to check.

Method

Weights = parameters × bytes per parameter (Q4_K_M 0.58, Q8_0 1.06, effective averages including scales and unquantised embedding and output layers). KV cache = 2 × layers × KV heads × head dimension × context × 2 bytes for a standard transformer; for the three designs above, only the layers that actually grow are counted at full length, sliding-window layers stop at their window, latent-attention layers count their compressed width, and the fixed recurrent state is added once. Overhead = 0.5 GB + 4% of weights. A tier "fits" when the total is at or under 95% of its memory. Architecture numbers come from each model's public config.json or model card:

• Llama 3.2 3B: meta-llama/Llama-3.2-3B config.json
• Qwen3.5 4B: Qwen/Qwen3.5-4B config.json
• Llama 3.1 8B: meta-llama/Llama-3.1-8B config.json
• Qwen3 8B: Qwen/Qwen3-8B config.json
• Granite 4.2 8B: ibm-granite/granite-4.2-8b config.json
• Qwen3.5 9B: Qwen/Qwen3.5-9B config.json
• Gemma 4 12B: google/gemma-4-12B-it config.json
• Gemma 3 12B: google/gemma-3-12b-it config.json
• Ministral 3 14B: mistralai/Ministral-3-14B-Instruct-2512 config.json
• Qwen3 14B: Qwen/Qwen3-14B config.json
• Phi-4 14B: microsoft/phi-4 config.json
• gpt-oss 20B: openai/gpt-oss-20b config.json
• Devstral Small 2 24B: mistralai/Devstral-Small-2-24B-Instruct-2512 config.json
• Mistral Small 3.1 24B: mistralai/Mistral-Small-3.1-24B-Instruct-2503 config.json
• Gemma 4 26B-A4B (MoE): google/gemma-4-26B-A4B-it config.json and model card
• Gemma 3 27B: google/gemma-3-27b-it config.json
• Qwen3.6 27B: Qwen/Qwen3.6-27B config.json
• Qwen3.8 27B: Qwen/Qwen3.8-27B config.json
• Granite 4.2 30B: ibm-granite/granite-4.2-30b config.json
• Muse Glimmer 30B: meta-models/Muse-Glimmer-30B config.json
• Qwen3 30B-A3B (MoE): Qwen/Qwen3-30B-A3B config.json
• GLM-4.7 Flash 30B-A3B (MoE): zai-org/GLM-4.7-Flash config.json and model card
• Gemma 4 31B: google/gemma-4-31B-it config.json
• Nemotron 3.5 Lightning 30B-A3B (MoE): nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 config.json and model card
• Qwen3 32B: Qwen/Qwen3-32B config.json
• Qwen2.5 Coder 32B: Qwen/Qwen2.5-Coder-32B-Instruct config.json
• DeepSeek-R1 Distill Qwen 32B: deepseek-ai/DeepSeek-R1-Distill-Qwen-32B config.json
• Qwen3.6 35B-A3B (MoE): Qwen/Qwen3.6-35B-A3B config.json and model card
• Llama 3.3 70B: meta-llama/Llama-3.3-70B-Instruct config.json
• DeepSeek-R1 Distill Llama 70B: deepseek-ai/DeepSeek-R1-Distill-Llama-70B config.json
• Nemotron 3.5 Bolt 74B-A9B (MoE): nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 config.json and model card
• Qwen3-Coder-Next 80B-A3B (MoE): Qwen/Qwen3-Coder-Next config.json and model card
• gpt-oss 120B: openai/gpt-oss-120b config.json
• Mistral Small 4 119B (MoE): mistralai/Mistral-Small-4-119B-2603 config.json and model card