What LLMs can a Mac Studio M2 Ultra run?

192 GB unified 800 GB/s 2023 Apple unified memory

Mac Studio M2 Ultra (192 GB) addresses 144 GB — 75% of its 192 GB of unified memory, which is what macOS hands the GPU by default at 800 GB/s. Of the 16 open models tracked on this site, it runs 16 at Q4_K_M with 8k of context. The largest is gpt-oss 120B, needing about 71.7 GB and generating an estimated 189 tokens per second — faster than you can read.

The quiet way to hold a very large model, as long as you are patient with long prompts.

What a Mac Studio M2 Ultra runs, model by model

Every model at Q4_K_M with 8k of context, the setting most people actually use. This card has 136.8 GB to spend once the 5% safety margin comes off its 144 GB, and it reads that memory at 800 GB/s. Those two numbers decide the last two columns: the first says what loads, the second says how fast it answers. Both are estimates from stated formulas, and the VRAM calculator shows the working.

On this card that means anything at or under 136.8 GB counts as fitting, and anything above 122.4 GB is marked tight: it loads, but there is little left for the conversation, and a desktop on the same 192 GB can push it over.

One caveat specific to Apple silicon: these tokens-per-second figures describe generation, which unified memory handles well. Prompt processing — reading what you send before the first token comes back — is markedly slower than on an equivalent Nvidia card, and that is what you feel when you paste in a long document. Short prompts feel fast on this machine; 30,000-token ones do not.

The model to actually run on it

gpt-oss 120B is the best use of this card: 71.7 GB of the 144 GB available, an estimated 189 tokens per second, faster than you can read, and 131,072 tokens of context still available. It is a mixture-of-experts model, so all 117B parameters sit in memory but only 5.1B are read per token — which is why it is quick for its size.

It also fits at Q8_0, in about 130.1 GB, which is worth taking whenever the memory allows: Q8 is near-lossless where Q4 costs a little accuracy.

How much context actually fits

Model size is the question people ask; context is the one that bites. The KV cache grows linearly with the conversation, so a model that loads comfortably can still run out of memory halfway through a long document. On 144 GB, the context you get is whatever the weights leave behind — which is why two cards that run the same model can feel completely different in use. These are the largest contexts this card holds, at Q4_K_M weights.

Quantising the KV cache to Q8 roughly doubles what you can hold — gpt-oss 120B on this card goes from 131,072 to 131,072 tokens — at a quality cost most people never notice. Capped at 128k tokens here; a model's own architecture may stop lower, and sliding-window models such as Gemma 3 use less KV than the formula assumes.

Where this card stops

Nothing in this list is out of reach. Every model tracked here fits at Q4_K_M with 8k of context, so the ceiling you will meet is context length and batch size, not parameter count.

The honest take

Unified memory means the GPU can address most of the machine's RAM, so a fully specified M2 Ultra holds models that need a rack of Nvidia hardware. Generation is respectable at 800 GB/s — between a 3090 and a 4090 — but prompt processing is markedly slower than on Nvidia, so long documents take a visible pause before the first token. For chat with a big model, it is excellent; for feeding it 30,000 tokens at a time, less so.

What it is good at

• Up to 192 GB of unified memory: 70B models at Q8, or 120B mixture-of-experts models, on a desktop machine.
• 800 GB/s with near-silent operation and a fraction of the power of an equivalent Nvidia build.
• MLX and llama.cpp are both well supported on Apple silicon.

What it is not

• Prompt processing is much slower than Nvidia. Long-context work feels different, not just slower.
• CUDA-only tooling does not run, which rules out a good deal of the fine-tuning and image-generation ecosystem.
• Memory is fixed at purchase; there is no upgrade path.

The thing people get wrong: You do not get all 192 GB. macOS caps GPU-addressable memory at roughly 75% of system RAM by default, so plan around about 144 GB unless you raise the limit yourself. Every fit calculation on this page uses the 75% figure.

Buy or rent

A genuine alternative to a multi-GPU build if you want silence, low power and very large models, and you do not need CUDA. If you need CUDA, the question is settled for you.

What matters is the memory upgrade rather than the machine, and Apple prices that differently in every configuration. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.

Nodegrove is the rented side of that comparison: a workspace that stays saved, with a GPU attached only while you are using it. It is the right answer when your hours are low or your needs change; buying is the right answer when they are high and stable.

Compared with the alternatives

• MacBook Pro M4 Max — 96 GB · 546 GB/s · best fit gpt-oss 120B at ~129 tok/s. A laptop that runs models a desktop GPU cannot hold.
• RTX PRO 6000 Blackwell — 96 GB · 1,792 GB/s · best fit gpt-oss 120B at ~424 tok/s. Ninety-six gigabytes and 5090-class bandwidth on one card.
• RTX 6000 Ada — 48 GB · 960 GB/s · best fit Qwen3 32B at ~35 tok/s. Forty-eight gigabytes in a normal computer, at 300 watts, without the noise.

All 13 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.

Questions

Can a Mac Studio M2 Ultra run a 70B model?

Yes. Llama 3.3 70B at Q4_K_M with 8k of context needs about 45.8 GB, inside the 144 GB this card can address, and it generates an estimated 14 tokens per second — usable, a little slow.

What is the best model to run on a Mac Studio M2 Ultra?

gpt-oss 120B. At Q4_K_M and 8k context it needs about 71.7 GB of the 144 GB available and generates an estimated 189 tokens per second, which is faster than you can read. It also fits at Q8, at about 130.1 GB, which is worth taking when it fits.

How many tokens per second does a Mac Studio M2 Ultra generate?

It depends entirely on the size of the model, because generating a token means reading every active weight out of memory. At 800 GB/s and 70% efficiency, this card produces an estimated 121 tokens per second on an 8B model at Q4, and about 189 on the largest model it holds, gpt-oss 120B. Reading speed is roughly 5 to 8 words per second, so anything above 25 feels immediate.

Is 144 GB enough for running LLMs locally?

It runs 16 of the 16 models tracked here at Q4_K_M with 8k of context, up to 117B parameters. The honest test is not the model list but the context: gpt-oss 120B on this card holds about 131,072 tokens before memory runs out.

Mac Studio M2 Ultra or MacBook Pro M4 Max for local models?

This card holds more: 144 GB against 96 GB, so it runs 16 of these models to the MacBook Pro M4 Max's 16. On speed, this card is faster: 800 GB/s against 546 GB/s, and bandwidth is what sets chat speed.

How much of a 192 GB Mac Studio can the GPU actually use?

About 75% by default, so roughly 144 GB, because macOS reserves the rest for the system. The limit can be raised with the iogpu.wired_limit_mb setting, at the risk of starving the operating system. Every figure on this page uses the 75% default.

Mac Studio or an Nvidia build for local LLMs?

The Mac holds far larger models for the money and does it silently at low power. The Nvidia build processes long prompts much faster and runs the CUDA-only half of the ecosystem, which includes most fine-tuning and image tooling. If you chat with big models, the Mac is excellent; if you feed them long documents or train anything, it is not.

• Memory: weights (parameters × bytes per parameter) + KV cache (2 × layers × KV heads × head dim × context × bytes) + 0.5 GB runtime + 4% of weights. Full derivation in the VRAM calculator.
• Speed: 70% × bandwidth ÷ active weight bytes, single stream, no batching, prompt processing excluded. See the tokens-per-second estimator.
• Card specification: 192 GB, 800 GB/s — manufacturer figures. Apple unified memory: macOS makes roughly 75% of system RAM addressable by the GPU by default, and every figure here uses that share.
• Model architecture values come from each model's published config; the self-hosted LLM guide explains what each one changes.