RTX PRO 4000 Blackwell SFF

Gemma 4 12B is the best use of this card: 8.2 GB of the 24 GB available, an estimated 43 tokens per second, comfortable for chat, and 131,072 tokens of context still available.

How this is chosen, since no benchmark is quoted: models are compared by size, a mixture-of-experts model counting at the geometric mean of its total and active parameters (a rule of thumb, not a measurement). The pick is the biggest class that fits with memory to spare and answers at 25 tokens per second or better, and within that class the most recent general-purpose release; coding and reasoning specialists are listed above but not recommended by default.

Qwen3.6 35B-A3B (MoE) is the largest model the card will hold, at 22.4 GB, but it is the wrong daily driver: filling 93% of memory leaves room for only 25,600 tokens of conversation. More parameters are not worth a context window that runs out mid-document. It is a mixture-of-experts model reading only 3B parameters per token, so it is the faster of the two at an estimated 174 tokens per second; what it gives up is the depth of a dense model that reads all of its weights for every token.

It also fits at Q8_0, in about 14.2 GB, which is worth taking whenever the memory allows: Q8 is near-lossless where Q4 costs a little accuracy.

Quantising the KV cache to Q8 roughly doubles what you can hold, at a quality cost most people never notice: Qwen3 32B on this card goes from 9,216 to 18,432 tokens. Capped at 128k tokens, or at the model's own native window where that is smaller, marked "model limit": free memory beyond that point buys nothing. Rows that hold far more context than their size suggests are hybrid, sliding-window or latent-attention models, which cache only a fraction of what a standard transformer does; each model page shows the working.

The first model out of reach is Llama 3.3 70B: about 45.8 GB at Q4_K_M and 8k context, against 24 GB of usable memory. Dropping to Q3_K_M would need about 37.7 GB, which still does not fit. Offloading the remainder to system RAM works and is ten to fifty times slower; it is a way to see a model run, not a way to use one.

You rarely buy this card on its own; it comes inside small workstations and AI boxes. Judge those by what 24 GB at 432 GB/s runs, which this page computes. For bigger models, renting a larger card for the hours you need it is the alternative to a second box.

Workstation cards are sold on quotes rather than shelf prices, so there is no single number to publish. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator, which already knows this card draws 70 W under load. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.

• RTX PRO 4000 Blackwell: 24 GB · 672 GB/s · best fit Muse Glimmer 30B at ~27 tok/s. Twenty-four gigabytes on a single-slot, 145 W workstation card.
• RTX 4090: 24 GB · 1,008 GB/s · best fit Muse Glimmer 30B at ~41 tok/s. The card most local-model advice is implicitly written for.
• RTX 5060 Ti: 16 GB · 448 GB/s · best fit Gemma 4 12B at ~45 tok/s. The newest 16 GB GeForce below the high end, and a clear step up from the 4060 Ti it replaces.

How it stacks up against the other personal AI computers, from the Ghost Core to an RTX PRO 6000 workstation: personal AI computers compared.

All 39 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.