How this is calculated
Weights = parameters × bytes per parameter for the chosen quantisation (FP16 2.00, Q8 1.06, Q6_K 0.82, Q5_K_M 0.71, Q4_K_M 0.58, Q3_K_M 0.47; effective averages that include scales and the unquantised embedding and output layers). KV cache = 2 × layers × KV heads × head dim × context × bytes per value. Overhead = 0.5 GB runtime context plus 4% of weights. "Fits" means at or under 95% of the card, leaving room for the desktop and the runtime. Mixture-of-experts models load every expert, so total parameters are used. Architecture numbers come from each model's public config; the list says which.
When the estimate is wrong, and in which direction
- Flash attention and paged KV reduce real usage below the estimate, sometimes by a few gigabytes at long context.
- Batch size above one multiplies the KV cache. This tool assumes a single conversation.
- Vision and audio inputs add encoder memory the tool does not model.
- Sliding-window architectures such as Gemma 3 use less KV than the formula suggests.
- The desktop on the same card takes 0.5 to 2 GB. The 95% "fits" threshold assumes the card is otherwise free; on a desktop with a display attached, treat a tight fit as a maybe.
For the full background, read the self-hosted LLM guide. For a table of popular models at common settings, see which models fit on 16 to 96 GB. If you have a specific card in mind, the can-I-run-it checker answers the pairing and says what to change when it does not fit.