RTX PRO 6000 Blackwell
An RTX PRO 6000 Blackwell runs 34 of the 34 open models tracked here at Q4_K_M with 8k context, computed on 16 October 2026. The largest is Mistral Small 4 119B (MoE), needing 72.5 GB and generating at most 333 tokens per second.
- Memory
- 96 GB VRAM
- Bandwidth
- 1,792 GB/s
- Board power
- 600 W
- Released
- 2025
- Availability
- Workstation card
Open the RTX PRO 6000 Blackwell in the LLM checker
What an RTX PRO 6000 Blackwell runs, model by model
RTX PRO 6000 Blackwell (96 GB) addresses 96 GB at 1,792 GB/s. Every model below is at Q4_K_M with 8k of context, the setting most people actually use. Once the 5% safety margin comes off, the card has 91.2 GB to spend: memory decides what loads, bandwidth how fast it answers. Both are estimates from stated formulas, and the VRAM calculator shows the working.
On this card a model fits at or under 91.2 GB, and above 81.6 GB it is tight: it loads, but there is little left for the conversation, and a desktop on the same 96 GB can push it over. Tokens per second are single-stream estimates from memory bandwidth; ≤ marks a ceiling real runtimes fall well below.
Computed 16 October 2026The model to actually run on it
Llama 3.3 70B is the best use of this card: 45.8 GB of the 96 GB available, an estimated 31 tokens per second, comfortable for chat, and 131,072 tokens of context still available.
How this is chosen, since no benchmark is quoted: models are compared by size, a mixture-of-experts model counting at the geometric mean of its total and active parameters (a rule of thumb, not a measurement). The pick is the biggest class that fits with memory to spare and answers at 25 tokens per second or better, and within that class the most recent general-purpose release; coding and reasoning specialists are listed above but not recommended by default.
Mistral Small 4 119B (MoE) is the largest model the card will hold, at 72.5 GB, with room for 131,072 tokens of context. It is a mixture-of-experts model reading only 6.5B parameters per token, so it is the faster of the two at an estimated 333 tokens per second; what it gives up is the depth of a dense model that reads all of its weights for every token.
It also fits at Q8_0, in about 81 GB, which is worth taking whenever the memory allows: Q8 is near-lossless where Q4 costs a little accuracy.
How much context actually fits
Model size is the question people ask; context is the one that bites. The KV cache grows linearly with the conversation, so a model that loads comfortably can still run out of memory halfway through a long document. On 96 GB, the context you get is whatever the weights leave behind, which is why two cards that run the same model can feel completely different in use. These are the largest contexts this card holds, at Q4_K_M weights.
| Model | Max context, FP16 KV | With Q8 KV cache |
|---|---|---|
| Llama 3.3 70B | 131,072 tokens | 131,072 tokens |
| Qwen3 32B | 40,960 tokens · model limit | 40,960 tokens · model limit |
| Gemma 4 31B | 131,072 tokens | 131,072 tokens |
| Muse Glimmer 30B | 131,072 tokens | 131,072 tokens |
Every model in this table already reaches its limit at FP16, so quantising the KV cache buys nothing here. Capped at 128k tokens, or at the model's own native window where that is smaller, marked "model limit": free memory beyond that point buys nothing. Rows that hold far more context than their size suggests are hybrid, sliding-window or latent-attention models, which cache only a fraction of what a standard transformer does; each model page shows the working.
Where this card stops
Nothing in this list is out of reach. Every model tracked here fits at Q4_K_M with 8k of context, so the ceiling you will meet is context length and batch size, not parameter count.
Buy or rent
Buy only if you need 96 GB regularly and locally. For the occasional 70B run, renting the same class of card for a handful of hours a month is not close.
Workstation cards are sold on quotes rather than shelf prices, so there is no single number to publish. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator, which already knows this card draws 600 W under load. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.
Rented, an RTX PRO 6000 (the Server Edition clouds run) costs from $1.69 an hour (RunPod Community, third-party host), or $1.80 from a provider that runs the machine (Nebius), on demand, before tax, read 19 October 2026. Every provider's price is in the cloud GPU price guide.
The honest take
Ninety-six gigabytes and 5090-class bandwidth on one card.
This is the first single card that runs a 70B model at Q8, or a 120B mixture-of-experts model at its native precision, without splitting anything. It pairs 96 GB with 1792 GB/s, the same bandwidth as a 5090, so unlike previous big-memory workstation cards it is not slow for its size. The workstation edition draws 600 W; a 300 W Max-Q variant trades clocks for a machine you can sit next to.
What it is good at
- 96 GB removes almost every capacity decision at the sizes open models currently ship in.
- 1792 GB/s means large models still generate at a comfortable pace, not a crawl.
- One card, one memory space: no tensor parallelism to configure, no split-brain debugging.
What it is not
- 600 W in the workstation edition, which dictates the power supply and the room.
- Priced as professional hardware; the money buys capacity, and the same capacity rents by the hour.
- Overkill for anything under 70B, where a 5090 does the same work at the same speed.
The thing people get wrong: It shares the 5090's 1792 GB/s. For any model that fits on both cards, they generate at the same speed. You are paying purely for the extra 64 GB of address space, not for a faster card.
Compared with the alternatives
- RTX 5090: 32 GB · 1,792 GB/s · best fit Muse Glimmer 30B at ~73 tok/s. The first consumer card whose memory bandwidth changes what is comfortable.
- RTX 6000 Ada: 48 GB · 960 GB/s · best fit Muse Glimmer 30B at ~39 tok/s. Forty-eight gigabytes in a normal computer, at 300 watts, without the noise.
- A100: 80 GB · 2,039 GB/s · best fit Llama 3.3 70B at ~35 tok/s. The old datacenter workhorse, still fast where it counts, and cheap to rent.
How it stacks up against the other personal AI computers, from the Ghost Core to an RTX PRO 6000 workstation: personal AI computers compared.
All 39 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.
Questions
- Can an RTX PRO 6000 Blackwell run a 70B model?
- Yes. Llama 3.3 70B at Q4_K_M with 8k of context needs about 45.8 GB, inside the 96 GB this card can address, and it generates an estimated 31 tokens per second, which is comfortable for chat.
- What is the best model to run on an RTX PRO 6000 Blackwell?
- Llama 3.3 70B. At Q4_K_M and 8k context it needs about 45.8 GB of the 96 GB available and generates an estimated 31 tokens per second, which is comfortable for chat. It also fits at Q8, at about 81 GB, which is worth taking when it fits.
- How many tokens per second does an RTX PRO 6000 Blackwell generate?
- It depends entirely on the size of the model, because generating a token means reading every active weight out of memory. At 1,792 GB/s and 70% efficiency, this card produces an estimated 270 tokens per second on an 8B model at Q4, and about 333 on the largest model it holds, Mistral Small 4 119B (MoE). Reading speed is roughly 5 to 8 words per second, so anything above 25 feels immediate. Treat 270 as a ceiling on 1,792 GB/s rather than a measurement; the higher a figure, the further real runtimes fall below it.
- Is 96 GB enough for running LLMs locally?
- It runs 34 of the 34 models tracked here at Q4_K_M with 8k of context, up to 119B parameters. The honest test is not the model list but the context: Llama 3.3 70B on this card holds about 131,072 tokens before memory runs out.
- RTX PRO 6000 Blackwell or RTX 5090 for local models?
- This card holds more: 96 GB against 32 GB, so it runs 34 of these models to the RTX 5090's 28. On speed, they are identical, and bandwidth is what sets chat speed.
- Can the RTX PRO 6000 run a 70B model at full precision?
- Not at FP16, which would need roughly 140 GB for the weights alone. At Q8, which is near-lossless in practice, a 70B model fits with room for a long context, and that is the first time a single card has been able to do it.
- RTX PRO 6000 or RTX 5090?
- They read memory at the same speed, 1,792 GB/s, so for any model that fits both they generate at the same rate. The only thing the extra money buys is the 96 GB address space, which matters if you run 70B models or very long contexts and is wasted if you do not.
- Memory: weights (parameters × bytes per parameter) + KV cache (2 × layers × KV heads × head dim × context × bytes for a standard transformer; sliding-window, hybrid and latent-attention models counted as they actually cache) + 0.5 GB runtime + 4% of weights. Full derivation in the VRAM calculator.
- Speed: 70% × bandwidth ÷ active weight bytes, single stream, no batching, prompt processing excluded. A ceiling, not a measurement: the 21 of 34 figures marked ≤ are where real runtimes fall furthest below the number, because so few weights are read per token. See the tokens-per-second estimator.
- Card specification (manufacturer figures): 96 GB, 1,792 GB/s, 600 W; see NVIDIA's own page for this card.
- Hourly prices: each provider's own pricing page or price API, read 19 October 2026, as listed in the cloud GPU price guide. Power: the card's full board power at $0.20 a kWh, before the rest of the PC.
- Model architecture values come from each model's published config; the self-hosted LLM guide explains what each one changes.