Monthly email

Can I run it? How much VRAM an LLM needs on your GPU

Pick your graphics card and an open model and get a straight answer: does it fit, how fast will it be, and if it does not fit, exactly what to change.

Updated Free, and it runs in your browser

Model
Your GPU
Quantisation
KV cache
No

Qwen3-Coder-Next 80B-A3B (MoE) at Q4_K_M, 8,192 context, on RTX 4090

Needs
48.9 GB
Card has
24 GB
Short by
26.1 GB

What would work instead

  • No context length helps: the weights alone are 48.6 GB before a single token of conversation.
  • The smallest card here that runs it exactly as asked: A100, 80 GB usable.
  • The biggest model your card does run at these settings, counting mixture-of-experts models at their dense equivalent: Qwen2.5 Coder 32B, 22.4 GB at ~37 tokens/s.

Or rent a bigger card

Qwen3-Coder-Next 80B-A3B (MoE) at Q4_K_M with 8,192 tokens of context needs 48.9 GB, which fits an 80 GB cloud GPU. Cloud GPU prices compared, with the date each price was read.

79.7B parameters, 3B active per token · 1,008 GB/s · All 512 experts stay in memory; 3B are active per token, so it answers like a small model once it is loaded.

Computed 16 October 2026Weights + KV cache + overhead, against 95% of the card · speed from memory bandwidth

How the answer is worked out

Three things have to share your card's memory, and only one of them is the model:

  • The weights: parameters multiplied by the bytes each one takes at your chosen quantisation. This is the big number and the one people already know.
  • The KV cache: the conversation so far, held in memory so the model does not re-read it every token. It grows linearly with context, and on long documents it can rival the weights.
  • Runtime overhead: roughly half a gigabyte of context and buffers, plus about 4% of the weights for activations.

The total is compared against 95% of what your card can address, leaving the rest for your desktop and for memory that fragments rather than packs. Mixture-of-experts models load every expert, so their full parameter count counts against memory even though only the active experts are read per token, which is why they are large and fast at the same time.

Speed is a separate question with a separate formula: generating a token means reading every active weight out of memory, so tokens per second is bandwidth divided by active weight size, at 70% efficiency. The speed estimator does that one on its own, and the VRAM calculator breaks the memory into its three parts.

When the answer is no

Most tools stop at the word "no". This one lists what would change it, with the number attached to each option, because the fixes are not equally good:

  • A smaller quantisation is the first lever and the cheapest. Q8 is near-lossless, Q5 and Q4 cost a little, Q3 costs enough that a smaller model at Q4 is usually the better trade.
  • A shorter context is free if you were not using it. If you were, it is the most painful cut on the list, because it changes what the model can do rather than how well it does it.
  • Quantising the KV cache to Q8 roughly halves the cache and is the option people forget. It is often the difference between a tight fit and a comfortable one.
  • A larger card is the honest answer when the gap is wide. The checker names the smallest one that works rather than pointing at the top of the range.
  • A different model is often the right call. A well-chosen smaller model at good precision beats a larger one crushed into three bits.

If none of those appeal, the fifth option is not to own the card at all. Renting the memory for the hours you actually use it is cheaper below a certain number of hours per month, and the build vs rent calculator will tell you where that line falls for your own numbers.

Straight to your card

Every card in the checker has a page of its own: the full model list with fits and speeds, how much context is left once the weights load, where the card stops, and an honest verdict on whether it is worth owning.

Or approach it from the other side: all 39 cards compared, and which models fit on 8 to 96 GB.

Questions

How do I know if a model will fit on my GPU?
A model fits when its weights, its KV cache and the runtime's own overhead together stay inside your card's memory. The checker adds those three up for the exact model, quantisation and context length you choose, and compares the total against 95% of your card's memory. The remainder covers the desktop and fragmentation.
What happens if a model does not fit in VRAM?
It still runs, and it becomes unusable. The runtime keeps the overflow in system RAM and moves it across the PCIe bus for every token, which is ten to fifty times slower than reading from the card. A model that generates 40 tokens per second when it fits can drop to one or two when it does not.
Can an RTX 4090 run Llama 3.3 70B?
Not at Q4_K_M with 8k of context: it needs about 45.8 GB and a 4090 has 24 GB, so it is roughly 23 GB short. The checker lists what would work: a smaller quantisation, a shorter context, a larger card, or the biggest model a 4090 does run.
Does the tool account for Apple unified memory?
Yes. macOS makes about 75% of system RAM addressable by the GPU by default, so a 128 GB machine is treated as roughly 96 GB. Note that these figures describe generation speed; prompt processing on Apple silicon is markedly slower than on an equivalent Nvidia card.
Are these numbers measured?
No, and no page on this site pretends otherwise. Memory is a stated formula over published architecture values, and speed is memory bandwidth divided by the weights read per token at 70% efficiency. These are ceilings worked out from memory bandwidth, not measurements, and we do not state an error margin we have not measured. The faster the figure, the further real runtimes fall below it: when few weights are read per token, as with small models and mixture-of-experts models, the costs the formula leaves out (kernel launches, expert routing, KV cache reads, sampling) take a growing share of each token. On memory, flash attention and paged KV usually bring real use in under the estimate.
  • Memory: weights + KV cache + runtime overhead, the formula set out in the VRAM calculator. Architecture values come from each model's published config.
  • Speed: 70% × memory bandwidth ÷ active weight bytes, single stream, prompt processing and batching excluded.
  • Card memory and bandwidth are manufacturer specifications. Apple machines are counted at 75% of unified memory, the macOS default for GPU-addressable RAM.
  • Sliding-window, hybrid and latent-attention models are counted as they actually cache. Not modelled: flash attention and paged KV (which reduce real use), batch sizes above one (which multiply the cache), and the working memory of vision and audio encoders.

The numbers, once a month.

New open models and what they need, price moves, our newest measurements. One email a month; unsubscribe in one click.

Or follow new findings and data changes by RSS.

One email a month. Unsubscribe in one click; your address is used for nothing else. Privacy notice