Mac mini M4 Pro

One caveat specific to Apple silicon: these tokens-per-second figures describe generation, which unified memory handles well. Prompt processing (reading what you send before the first token comes back) is markedly slower than on an equivalent Nvidia card, and that is what you feel when you paste in a long document. Short prompts feel fast on this machine; 30,000-token ones do not.

Gemma 4 12B is the best use of this card: 8.2 GB of the 48 GB available, an estimated 27 tokens per second, comfortable for chat, and 131,072 tokens of context still available.

How this is chosen, since no benchmark is quoted: models are compared by size, a mixture-of-experts model counting at the geometric mean of its total and active parameters (a rule of thumb, not a measurement). The pick is the biggest class that fits with memory to spare and answers at 25 tokens per second or better, and within that class the most recent general-purpose release; coding and reasoning specialists are listed above but not recommended by default.

Qwen3.6 35B-A3B (MoE) is the largest model the card will hold, at 22.4 GB, with room for 131,072 tokens of context. It is a mixture-of-experts model reading only 3B parameters per token, so it is the faster of the two at an estimated 110 tokens per second; what it gives up is the depth of a dense model that reads all of its weights for every token.

If quality matters more than parameter count, Muse Glimmer 30B fits at Q8_0 in about 33.5 GB. Q8 is near-lossless; a smaller model at Q8 often beats a larger one squeezed into Q3.

Every model in this table already reaches its limit at FP16, so quantising the KV cache buys nothing here. Capped at 128k tokens, or at the model's own native window where that is smaller, marked "model limit": free memory beyond that point buys nothing. Rows that hold far more context than their size suggests are hybrid, sliding-window or latent-attention models, which cache only a fraction of what a standard transformer does; each model page shows the working.

The first model out of reach is Llama 3.3 70B: about 45.8 GB at Q4_K_M and 8k context, against 48 GB of usable memory. That is close enough to be worth a caveat: it is inside the raw 48 GB and only misses the 95% line this site uses for a fit. On a card with a display attached, treat it as no. On a headless card, with a shorter context, it runs. Dropping to Q3_K_M would need about 37.7 GB, which fits, though Q3 loses enough quality that a smaller model at Q4 is usually the better trade. Offloading the remainder to system RAM works and is ten to fifty times slower; it is a way to see a model run, not a way to use one.

Worth it second-hand or refurbished at 64 GB if you want a quiet machine that holds mid-sized models and you do not need CUDA. Bought new, the M5 Pro Mac mini has the same 64 GB ceiling and reads memory about an eighth faster. For occasional large models, renting a big card by the hour costs less than buying for the largest one.

What matters is the memory upgrade rather than the machine, and Apple prices that differently in every configuration. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.

• Mac mini M5 Pro: 48 GB · 307 GB/s · best fit Gemma 4 12B at ~31 tok/s. Apple's current Mac mini for local models: up to 64 GB at 307 GB/s.
• DGX Spark: 126 GB · 273 GB/s · best fit Nemotron 3.5 Bolt 74B-A9B (MoE) at ~37 tok/s. NVIDIA's small Linux box with 128 GB the GPU can share: large models fit, but they are read at about a quarter of an RTX 4090's speed.
• Ryzen AI Max+ 395: 96 GB · 256 GB/s · best fit Nemotron 3.5 Bolt 74B-A9B (MoE) at ~34 tok/s. AMD's 128 GB laptop and mini-PC chip: up to 96 GB for the GPU on Windows, and an x86 route to large mixture-of-experts models.

All 39 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.