RTX 5090
An RTX 5090 runs 28 of the 34 open models tracked here at Q4_K_M with 8k context, computed on 16 October 2026. The largest is Qwen3.6 35B-A3B (MoE), needing 22.4 GB and generating at most 721 tokens per second.
- Memory
- 32 GB VRAM
- Bandwidth
- 1,792 GB/s
- Board power
- 575 W
- Released
- 2025
- Availability
- Sold new
Open the RTX 5090 in the LLM checker
What an RTX 5090 runs, model by model
RTX 5090 (32 GB) addresses 32 GB at 1,792 GB/s. Every model below is at Q4_K_M with 8k of context, the setting most people actually use. Once the 5% safety margin comes off, the card has 30.4 GB to spend: memory decides what loads, bandwidth how fast it answers. Both are estimates from stated formulas, and the VRAM calculator shows the working.
On this card a model fits at or under 30.4 GB, and above 27.2 GB it is tight: it loads, but there is little left for the conversation, and a desktop on the same 32 GB can push it over. Tokens per second are single-stream estimates from memory bandwidth; ≤ marks a ceiling real runtimes fall well below.
Computed 16 October 2026The model to actually run on it
Muse Glimmer 30B is the best use of this card: 18.7 GB of the 32 GB available, an estimated 73 tokens per second, faster than you can read, and 131,072 tokens of context still available.
How this is chosen, since no benchmark is quoted: models are compared by size, a mixture-of-experts model counting at the geometric mean of its total and active parameters (a rule of thumb, not a measurement). The pick is the biggest class that fits with memory to spare and answers at 25 tokens per second or better, and within that class the most recent general-purpose release; coding and reasoning specialists are listed above but not recommended by default.
Qwen3.6 35B-A3B (MoE) is the largest model the card will hold, at 22.4 GB, with room for 131,072 tokens of context. It is a mixture-of-experts model reading only 3B parameters per token, so it is the faster of the two at an estimated 721 tokens per second; what it gives up is the depth of a dense model that reads all of its weights for every token.
If quality matters more than parameter count, Mistral Small 3.1 24B fits at Q8_0 in about 28.3 GB. Q8 is near-lossless; a smaller model at Q8 often beats a larger one squeezed into Q3.
How much context actually fits
Model size is the question people ask; context is the one that bites. The KV cache grows linearly with the conversation, so a model that loads comfortably can still run out of memory halfway through a long document. On 32 GB, the context you get is whatever the weights leave behind, which is why two cards that run the same model can feel completely different in use. These are the largest contexts this card holds, at Q4_K_M weights.
| Model | Max context, FP16 KV | With Q8 KV cache |
|---|---|---|
| Muse Glimmer 30B | 131,072 tokens | 131,072 tokens |
| Qwen3 32B | 37,888 tokens | 40,960 tokens · model limit |
| Gemma 4 31B | 123,904 tokens | 131,072 tokens |
| Granite 4.2 30B | 46,080 tokens | 93,184 tokens |
Quantising the KV cache to Q8 roughly doubles what you can hold, at a quality cost most people never notice: Gemma 4 31B on this card goes from 123,904 to 131,072 tokens. Capped at 128k tokens, or at the model's own native window where that is smaller, marked "model limit": free memory beyond that point buys nothing. Rows that hold far more context than their size suggests are hybrid, sliding-window or latent-attention models, which cache only a fraction of what a standard transformer does; each model page shows the working.
Where this card stops
The first model out of reach is Llama 3.3 70B: about 45.8 GB at Q4_K_M and 8k context, against 32 GB of usable memory. Dropping to Q3_K_M would need about 37.7 GB, which still does not fit. Offloading the remainder to system RAM works and is ten to fifty times slower; it is a way to see a model run, not a way to use one.
Buy or rent
The case for buying is strongest here if you run models daily and value the time. For occasional use, the power draw and the outlay both argue for renting a bigger card only when you need it.
NVIDIA lists the RTX 5090 from $1,999; what a shop charges may differ. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator, which already knows this card draws 575 W under load. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.
Rented, an RTX 5090 costs from $0.50 an hour (SaladCloud, home PCs), or $1.19 from a provider that runs the machine (RunPod), on demand, before tax, read 19 October 2026. Every provider's price is in the cloud GPU price guide.
Your own card draws power too: at 575 W and $0.20 a kWh, about $0.12 an hour. Take that off, and at the listed price the card has paid for itself after about 1,860 hours against RunPod, or 5,190 against SaladCloud. Over three years that makes renting cheaper below about 12 hours a week against the data-centre price, or 33 against SaladCloud; above it, buying wins. That leaves out the rest of the PC and what the card sells for later.
The honest take
The first consumer card whose memory bandwidth changes what is comfortable.
Thirty-two gigabytes is a modest capacity step; 1792 GB/s is not. The 5090 generates text roughly 78% faster than a 4090 on the same model, which moves 32B-class models from "quick" to "instant" and makes larger models at aggressive quantisation genuinely usable. It is also 575 W, which is a real constraint in a room you sit in.
What it is good at
- 1792 GB/s is the highest bandwidth outside the datacenter, and bandwidth is what sets chat speed.
- 32 GB clears the 24 GB cliff: 32B models fit at Q4 with long context, and Q5 becomes an option.
- GDDR7 and Blackwell compute make it the strongest consumer card for image and video generation as well.
What it is not
- 575 W. In a small room this is a heater, and it needs a power supply chosen for it.
- 32 GB still does not fit a 70B model at Q4 with useful context.
- Newest generation means the occasional runtime or driver rough edge that older cards no longer have.
The thing people get wrong: The 8 GB of extra memory over a 4090 is not why this card is faster. Text generation speed is bandwidth divided by weight size, and the 5090 moves 78% more memory per second. That single number explains almost all of the difference.
Compared with the alternatives
- RTX 4090: 24 GB · 1,008 GB/s · best fit Muse Glimmer 30B at ~41 tok/s. The card most local-model advice is implicitly written for.
- RTX PRO 6000 Blackwell: 96 GB · 1,792 GB/s · best fit Llama 3.3 70B at ~31 tok/s. Ninety-six gigabytes and 5090-class bandwidth on one card.
- RTX 3090: 24 GB · 936 GB/s · best fit Muse Glimmer 30B at ~38 tok/s. The value benchmark for local models, and it has been for years.
How it stacks up against the other personal AI computers, from the Ghost Core to an RTX PRO 6000 workstation: personal AI computers compared.
All 39 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.
Questions
- Can an RTX 5090 run a 70B model?
- Not at Q4_K_M in one card. Llama 3.3 70B needs about 45.8 GB at 8k context and this card can address 32 GB. Your options are a smaller model, a harsher quantisation with very little context, splitting across two cards, or renting a larger one for the hours you need it.
- What is the best model to run on an RTX 5090?
- Muse Glimmer 30B. At Q4_K_M and 8k context it needs about 18.7 GB of the 32 GB available and generates an estimated 73 tokens per second, which is faster than you can read. If quality matters more than size, Mistral Small 3.1 24B fits at Q8 in about 28.3 GB.
- How many tokens per second does an RTX 5090 generate?
- It depends entirely on the size of the model, because generating a token means reading every active weight out of memory. At 1,792 GB/s and 70% efficiency, this card produces an estimated 270 tokens per second on an 8B model at Q4, and about 721 on the largest model it holds, Qwen3.6 35B-A3B (MoE). Reading speed is roughly 5 to 8 words per second, so anything above 25 feels immediate. Treat 270 as a ceiling on 1,792 GB/s rather than a measurement; the higher a figure, the further real runtimes fall below it.
- Is 32 GB enough for running LLMs locally?
- It runs 28 of the 34 models tracked here at Q4_K_M with 8k of context, up to 36B parameters. The honest test is not the model list but the context: Muse Glimmer 30B on this card holds about 131,072 tokens before memory runs out.
- RTX 5090 or RTX 4090 for local models?
- This card holds more: 32 GB against 24 GB, but both run the same 28 of these models at Q4_K_M with 8k of context, so the extra memory goes to longer context and higher precision. On speed, this card is faster: 1,792 GB/s against 1,008 GB/s, and bandwidth is what sets chat speed.
- Is the RTX 5090 worth it over a 4090 for AI?
- For text generation, yes, more than the specification sheet suggests: 1,792 GB/s against 1,008 GB/s is roughly 78% more memory read per second, and that is what sets generation speed. The 8 GB of extra capacity matters less than the bandwidth, though it does bring 32B models within reach at longer context.
- How much power does an RTX 5090 need for AI workloads?
- The board is rated at 575 W, and inference holds it near that for as long as the model is generating. Plan the power supply and the room around it. In a small space this is a meaningful amount of heat, which is a real argument for renting the occasional large run rather than owning it.
- Memory: weights (parameters × bytes per parameter) + KV cache (2 × layers × KV heads × head dim × context × bytes for a standard transformer; sliding-window, hybrid and latent-attention models counted as they actually cache) + 0.5 GB runtime + 4% of weights. Full derivation in the VRAM calculator.
- Speed: 70% × bandwidth ÷ active weight bytes, single stream, no batching, prompt processing excluded. A ceiling, not a measurement: the 17 of 28 figures marked ≤ are where real runtimes fall furthest below the number, because so few weights are read per token. See the tokens-per-second estimator.
- Card specification (manufacturer figures): 32 GB, 1,792 GB/s, 575 W; see NVIDIA's own page for this card. The bandwidth figure: NVIDIA's GeForce comparison page.
- Hourly prices: each provider's own pricing page or price API, read 19 October 2026, as listed in the cloud GPU price guide. Power: the card's full board power at $0.20 a kWh, before the rest of the PC.
- Model architecture values come from each model's published config; the self-hosted LLM guide explains what each one changes.