RTX 4080 SUPER

Gemma 4 12B is the best use of this card: 8.2 GB of the 16 GB available, an estimated 74 tokens per second, faster than you can read, and 131,072 tokens of context still available.

How this is chosen, since no benchmark is quoted: models are compared by size, a mixture-of-experts model counting at the geometric mean of its total and active parameters (a rule of thumb, not a measurement). The pick is the biggest class that fits with memory to spare and answers at 25 tokens per second or better, and within that class the most recent general-purpose release; coding and reasoning specialists are listed above but not recommended by default.

gpt-oss 20B is the largest model the card will hold, at 13.4 GB, with room for 81,920 tokens of context. It is a mixture-of-experts model reading only 3.6B parameters per token, so it is the faster of the two at an estimated 247 tokens per second; what it gives up is the depth of a dense model that reads all of its weights for every token.

It also fits at Q8_0, in about 14.2 GB, which is worth taking whenever the memory allows: Q8 is near-lossless where Q4 costs a little accuracy.

Quantising the KV cache to Q8 roughly doubles what you can hold, at a quality cost most people never notice: Ministral 3 14B on this card goes from 37,888 to 75,776 tokens. Capped at 128k tokens, or at the model's own native window where that is smaller, marked "model limit": free memory beyond that point buys nothing. Rows that hold far more context than their size suggests are hybrid, sliding-window or latent-attention models, which cache only a fraction of what a standard transformer does; each model page shows the working.

The first model out of reach is Devstral Small 2 24B: about 16.3 GB at Q4_K_M and 8k context, against 16 GB of usable memory. Dropping to Q3_K_M would need about 13.6 GB, which fits, though Q3 loses enough quality that a smaller model at Q4 is usually the better trade. Offloading the remainder to system RAM works and is ten to fifty times slower; it is a way to see a model run, not a way to use one.

Do not buy one for language models: a used 3090 holds more and reads it faster, and the newer 5070 Ti reads the same 16 GB about a fifth faster. If you already own one, it is a capable 16 GB card; rent a larger card for the models it cannot hold.

Second-hand pricing is the whole argument for this card, and it moves week to week, so printing a number here would be worse than printing none. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator, which already knows this card draws 320 W under load. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.

• RTX 3090: 24 GB · 936 GB/s · best fit Muse Glimmer 30B at ~38 tok/s. The value benchmark for local models, and it has been for years.
• RTX 5080: 16 GB · 960 GB/s · best fit Gemma 4 12B at ~97 tok/s. A 16 GB card with 3090-class bandwidth: quick on what fits, and it fits no more than a 5060 Ti 16 GB.
• RTX 5070 Ti: 16 GB · 896 GB/s · best fit Gemma 4 12B at ~90 tok/s. The 5080's memory bus and 16 GB, read about 7% more slowly.

All 39 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.