MacBook Pro M4 Max

One caveat specific to Apple silicon: these tokens-per-second figures describe generation, which unified memory handles well. Prompt processing (reading what you send before the first token comes back) is markedly slower than on an equivalent Nvidia card, and that is what you feel when you paste in a long document. Short prompts feel fast on this machine; 30,000-token ones do not.

Nemotron 3.5 Bolt 74B-A9B (MoE) is the best use of this card: 45.9 GB of the 96 GB available, an estimated 73 tokens per second, faster than you can read, and 131,072 tokens of context still available. It is a mixture-of-experts model, so all 74.9B parameters sit in memory but only 9B are read per token, which is why it is quick for its size.

How this is chosen, since no benchmark is quoted: models are compared by size, a mixture-of-experts model counting at the geometric mean of its total and active parameters (a rule of thumb, not a measurement). The pick is the biggest class that fits with memory to spare and answers at 25 tokens per second or better, and within that class the most recent general-purpose release; coding and reasoning specialists are listed above but not recommended by default.

Mistral Small 4 119B (MoE) is the largest model the card will hold, at 72.5 GB, with room for 131,072 tokens of context. It is a mixture-of-experts model reading only 6.5B parameters per token, so it is the faster of the two at an estimated 101 tokens per second; what it gives up is the depth of a dense model that reads all of its weights for every token.

If quality matters more than parameter count, Llama 3.3 70B fits at Q8_0 in about 81 GB. Q8 is near-lossless; a smaller model at Q8 often beats a larger one squeezed into Q3.

Every model in this table already reaches its limit at FP16, so quantising the KV cache buys nothing here. Capped at 128k tokens, or at the model's own native window where that is smaller, marked "model limit": free memory beyond that point buys nothing. Rows that hold far more context than their size suggests are hybrid, sliding-window or latent-attention models, which cache only a fraction of what a standard transformer does; each model page shows the working.

Nothing in this list is out of reach. Every model tracked here fits at Q4_K_M with 8k of context, so the ceiling you will meet is context length and batch size, not parameter count.

If you were buying the laptop anyway, the memory upgrade is the cheapest large-model capacity in this list. If you were not, it is an expensive way to buy GPU memory.

What matters is the memory upgrade rather than the machine, and Apple prices that differently in every configuration. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.

• Mac Studio M2 Ultra: 144 GB · 800 GB/s · best fit Muse Glimmer 30B at ~32 tok/s. The quiet way to hold a very large model, as long as you are patient with long prompts.
• RTX 4090: 24 GB · 1,008 GB/s · best fit Muse Glimmer 30B at ~41 tok/s. The card most local-model advice is implicitly written for.
• RTX 5090: 32 GB · 1,792 GB/s · best fit Muse Glimmer 30B at ~73 tok/s. The first consumer card whose memory bandwidth changes what is comfortable.

All 39 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.