Already know which card and which model? The can-I-run-it checker answers that pairing directly. Every card below is measured the same way: each of the 16 open models tracked on this site, at Q4_K_M with 8k of context, against the memory the card can actually address. Nothing here is benchmarked — memory comes from a stated formula, speed from memory bandwidth, and both say so. Pick a card for the page with its full working, or read the two numbers that decide everything first.
The only two numbers that matter
Memory decides what you can run. A model has to fit, in full, along with its KV cache — and if it does not fit, the answer is not "slower", it is ten to fifty times slower, because the shortfall spills to system RAM. Fit is binary, and it is the first filter.
Bandwidth decides how fast it answers. Generating one token means reading every active weight out of memory, so the ceiling is bandwidth divided by weight size. This is why a 1008 GB/s card is only about 7% faster than a 936 GB/s one for chat, whatever the rest of the specification says, and why the 16 GB RTX 4060 Ti at 288 GB/s generates more slowly than the cheaper 12 GB RTX 3060.
Compute — cores, TFLOPs, the number in most reviews — barely affects chat speed. It matters for prompt processing, for image and video generation, and for serving many people at once. For one person typing questions at a model, it is close to irrelevant.
Every card, compared
| Card | Usable | Bandwidth | Runs | Largest model it holds |
|---|---|---|---|---|
| RTX 3060 | 12 GB | 360 GB/s | 6/16 | Qwen3 14B· 29 tok/s |
| RTX 4070 | 12 GB | 504 GB/s | 6/16 | Qwen3 14B· 41 tok/s |
| RTX 4060 Ti | 16 GB | 288 GB/s | 7/16 | gpt-oss 20B· 97 tok/s |
| RTX 3090 | 24 GB | 936 GB/s | 13/16 | Qwen3 32B· 34 tok/s |
| RTX 4090 | 24 GB | 1,008 GB/s | 13/16 | Qwen3 32B· 37 tok/s |
| RTX 5090 | 32 GB | 1,792 GB/s | 13/16 | Qwen3 32B· 66 tok/s |
| L40S | 48 GB | 864 GB/s | 13/16 | Qwen3 32B· 32 tok/s |
| RTX 6000 Ada | 48 GB | 960 GB/s | 13/16 | Qwen3 32B· 35 tok/s |
| A100 | 80 GB | 2,039 GB/s | 16/16 | gpt-oss 120B· 483 tok/s |
| H100 SXM | 80 GB | 3,352 GB/s | 16/16 | gpt-oss 120B· 793 tok/s |
| MacBook Pro M4 Max | 96 GB of 128 | 546 GB/s | 16/16 | gpt-oss 120B· 129 tok/s |
| RTX PRO 6000 Blackwell | 96 GB | 1,792 GB/s | 16/16 | gpt-oss 120B· 424 tok/s |
| Mac Studio M2 Ultra | 144 GB of 192 | 800 GB/s | 16/16 | gpt-oss 120B· 189 tok/s |
"Usable" is the memory a runtime can address: full VRAM on a graphics card, and about 75% of system RAM on Apple silicon, which is the macOS default. "Runs" counts models that fit at Q4_K_M with 8k of context, at or under 95% of usable memory.
Consumer cards
What goes in a desktop you already own. Memory is the ceiling on what you can run; bandwidth is the ceiling on how fast it answers.
- RTX 3060 12 GB · 360 GB/s · 170 W · 2021 · Mostly second-hand
The cheapest card that still makes local models worth doing.
Runs 6 of 16 models · best fit: Qwen3 8B at ~53 tok/s - RTX 4070 12 GB · 504 GB/s · 200 W · 2023 · Sold new
Fast for its class, and capped by 12 GB.
Runs 6 of 16 models · best fit: Qwen3 8B at ~74 tok/s - RTX 4060 Ti 16 GB · 288 GB/s · 165 W · 2023 · Sold new
More memory than its neighbours, and less bandwidth than a card two years older.
Runs 7 of 16 models · best fit: gpt-oss 20B at ~97 tok/s - RTX 3090 24 GB · 936 GB/s · 350 W · 2020 · Mostly second-hand
The value benchmark for local models, and it has been for years.
Runs 13 of 16 models · best fit: Qwen3 30B-A3B (MoE) at ~342 tok/s - RTX 4090 24 GB · 1,008 GB/s · 450 W · 2022 · Mostly second-hand
The card most local-model advice is implicitly written for.
Runs 13 of 16 models · best fit: Qwen3 30B-A3B (MoE) at ~369 tok/s - RTX 5090 32 GB · 1,792 GB/s · 575 W · 2025 · Sold new
The first consumer card whose memory bandwidth changes what is comfortable.
Runs 13 of 16 models · best fit: Qwen3 32B at ~66 tok/s
Workstation cards
Large memory in a machine that lives under a desk, at power and noise levels a room can tolerate. You pay for the address space, not the speed.
- RTX 6000 Ada 48 GB · 960 GB/s · 300 W · 2022 · Workstation card
Forty-eight gigabytes in a normal computer, at 300 watts, without the noise.
Runs 13 of 16 models · best fit: Qwen3 32B at ~35 tok/s - RTX PRO 6000 Blackwell 96 GB · 1,792 GB/s · 600 W · 2025 · Workstation card
Ninety-six gigabytes and 5090-class bandwidth on one card.
Runs 16 of 16 models · best fit: gpt-oss 120B at ~424 tok/s
Datacenter cards
You meet these as a line on a rental invoice. They are here because they set the reference for what renting buys you.
- L40S 48 GB · 864 GB/s · 350 W · 2023 · Rented, not bought
The 48 GB card you meet as a rental line item, not as a purchase.
Runs 13 of 16 models · best fit: Qwen3 32B at ~32 tok/s - A100 80 GB · 2,039 GB/s · 2020 · Rented, not bought
The old datacenter workhorse, still fast where it counts, and cheap to rent.
Runs 16 of 16 models · best fit: Llama 3.3 70B at ~35 tok/s - H100 SXM 80 GB · 3,352 GB/s · 700 W · 2022 · Rented, not bought
The fastest memory here by a wide margin, and almost always more than one person needs.
Runs 16 of 16 models · best fit: Llama 3.3 70B at ~57 tok/s
Apple unified memory
The GPU addresses most of system RAM, so these hold models no single graphics card can. Generation is respectable; prompt processing is not.
- MacBook Pro M4 Max 96 of 128 GB usable · 546 GB/s · 2024 · Apple unified memory
A laptop that runs models a desktop GPU cannot hold.
Runs 16 of 16 models · best fit: gpt-oss 120B at ~129 tok/s - Mac Studio M2 Ultra 144 of 192 GB usable · 800 GB/s · 2023 · Apple unified memory
The quiet way to hold a very large model, as long as you are patient with long prompts.
Runs 16 of 16 models · best fit: gpt-oss 120B at ~189 tok/s
How to choose, in one paragraph
Decide the largest model you actually want to run, then buy the memory that holds it, then take whatever bandwidth comes with that memory. Going the other way — picking a fast card and hoping the model fits — is how people end up with a card that runs everything quickly except the thing they bought it for. If you are unsure which model you want, start from the VRAM calculator: it tells you the smallest card each model fits on. If you know the model and want to know how it will feel, the tokens-per-second estimator answers that. If you already know both and the question is whether to buy at all, the build vs rent calculator takes your own hours and electricity price.
And the honest version: most people over-buy. Hours per month decides the economics, and a card that idles is a card that has not paid for itself. Renting the class of card that runs gpt-oss 120B for a few hours a month costs less than owning almost anything on this page. Nodegrove exists for that side of the trade — a workspace that stays saved, with a GPU attached only while you use it — but the calculator will tell you honestly when buying wins.
- Card specifications are manufacturer figures: memory capacity, memory bandwidth and board power.
- Memory estimates: weights + KV cache + runtime overhead, per the formula in the VRAM calculator.
- Speed estimates: 70% × bandwidth ÷ active weight bytes, single stream, prompt processing and batching excluded.
- No prices appear on this page on purpose. They move weekly; the build vs rent calculator uses yours.