LLM VRAM calculator

Weights = parameters × bytes per parameter for the chosen quantisation (FP16 2.00, Q8 1.06, Q6_K 0.82, Q5_K_M 0.71, Q4_K_M 0.58, Q3_K_M 0.47; effective averages that include scales and the unquantised embedding and output layers). KV cache = 2 × layers × KV heads × head dim × context × bytes per value for a standard transformer. Three newer designs are counted the way they actually cache: sliding-window layers (Gemma, gpt-oss) stop at their window, hybrid models (Qwen3.5 and later, Nemotron) only grow a cache on their few full-attention layers and carry a small fixed state for the rest, and latent-attention models (Mistral Small 4, GLM Flash) cache one compressed vector per layer. Each model page says which applies. Overhead = 0.5 GB runtime context plus 4% of weights. "Fits" means at or under 95% of the card, leaving room for the desktop and the runtime. Mixture-of-experts models load every expert, so total parameters are used. Architecture numbers come from each model's public config; the list says which.

Generating a token reads every active weight once, so the ceiling is memory bandwidth divided by the bytes of active weights. We multiply by 0.7 as a stated allowance for kernel overhead, KV cache reads and sampling; it is an assumption, not a measured figure. Mixture-of-experts models only read their active experts, which is why they are fast for their size. These are ceilings worked out from memory bandwidth, not measurements, and we do not state an error margin we have not measured. The faster the figure, the further real runtimes fall below it: when few weights are read per token, as with small models and mixture-of-experts models, the costs the formula leaves out (kernel launches, expert routing, KV cache reads, sampling) take a growing share of each token. Prompt processing is not included; it is compute-bound and much faster per token. Batching is not included; it multiplies throughput for a server, not for one conversation. If the model does not fit the card, the number is meaningless, and the tool says so.