RX 9070 XT
An RX 9070 XT runs 12 of the 34 open models tracked here at Q4_K_M with 8k context, computed on 16 October 2026. The largest is gpt-oss 20B, needing 13.4 GB and generating at most 215 tokens per second.
- Memory
- 16 GB VRAM
- Bandwidth
- 640 GB/s
- Board power
- 304 W
- Released
- 2025
- Availability
- Sold new
Open the RX 9070 XT in the LLM checker
What an RX 9070 XT runs, model by model
RX 9070 XT (16 GB) addresses 16 GB at 640 GB/s. Every model below is at Q4_K_M with 8k of context, the setting most people actually use. Once the 5% safety margin comes off, the card has 15.2 GB to spend: memory decides what loads, bandwidth how fast it answers. Both are estimates from stated formulas, and the VRAM calculator shows the working.
On this card a model fits at or under 15.2 GB, and above 13.6 GB it is tight: it loads, but there is little left for the conversation, and a desktop on the same 16 GB can push it over. Tokens per second are single-stream estimates from memory bandwidth; ≤ marks a ceiling real runtimes fall well below.
Computed 16 October 2026The model to actually run on it
Gemma 4 12B is the best use of this card: 8.2 GB of the 16 GB available, an estimated 64 tokens per second, faster than you can read, and 131,072 tokens of context still available.
How this is chosen, since no benchmark is quoted: models are compared by size, a mixture-of-experts model counting at the geometric mean of its total and active parameters (a rule of thumb, not a measurement). The pick is the biggest class that fits with memory to spare and answers at 25 tokens per second or better, and within that class the most recent general-purpose release; coding and reasoning specialists are listed above but not recommended by default.
gpt-oss 20B is the largest model the card will hold, at 13.4 GB, with room for 81,920 tokens of context. It is a mixture-of-experts model reading only 3.6B parameters per token, so it is the faster of the two at an estimated 215 tokens per second; what it gives up is the depth of a dense model that reads all of its weights for every token.
It also fits at Q8_0, in about 14.2 GB, which is worth taking whenever the memory allows: Q8 is near-lossless where Q4 costs a little accuracy.
How much context actually fits
Model size is the question people ask; context is the one that bites. The KV cache grows linearly with the conversation, so a model that loads comfortably can still run out of memory halfway through a long document. On 16 GB, the context you get is whatever the weights leave behind, which is why two cards that run the same model can feel completely different in use. These are the largest contexts this card holds, at Q4_K_M weights.
| Model | Max context, FP16 KV | With Q8 KV cache |
|---|---|---|
| Gemma 4 12B | 131,072 tokens | 131,072 tokens |
| Qwen3 14B | 34,816 tokens | 40,960 tokens · model limit |
| Phi-4 14B | 16,384 tokens · model limit | 16,384 tokens · model limit |
| Ministral 3 14B | 37,888 tokens | 75,776 tokens |
Quantising the KV cache to Q8 roughly doubles what you can hold, at a quality cost most people never notice: Ministral 3 14B on this card goes from 37,888 to 75,776 tokens. Capped at 128k tokens, or at the model's own native window where that is smaller, marked "model limit": free memory beyond that point buys nothing. Rows that hold far more context than their size suggests are hybrid, sliding-window or latent-attention models, which cache only a fraction of what a standard transformer does; each model page shows the working.
Where this card stops
The first model out of reach is Devstral Small 2 24B: about 16.3 GB at Q4_K_M and 8k context, against 16 GB of usable memory. Dropping to Q3_K_M would need about 13.6 GB, which fits, though Q3 loses enough quality that a smaller model at Q4 is usually the better trade. Offloading the remainder to system RAM works and is ten to fifty times slower; it is a way to see a model run, not a way to use one.
Buy or rent
A sensible new card if 16 GB covers what you run and you are happy to check each tool for ROCm or Vulkan support. If you need 24 GB, an RX 7900 XTX or a used RTX 3090 holds more and reads it faster. For the occasional larger model, renting a bigger card by the hour costs less than buying up a tier.
Retail prices drift, and a figure written today would be wrong within a month, so this page does not carry one. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator, which already knows this card draws 304 W under load. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.
The honest take
AMD's RDNA 4 gaming card: 16 GB at 640 GB/s, with ROCm on Linux and Windows, and no CUDA.
It holds what any 16 GB card holds and reads it at 640 GB/s, about 1.4 times an RTX 5060 Ti 16 GB, so the same models have a higher speed ceiling. RDNA 4 adds FP8, which the RX 7000 series lacks, and AMD's ROCm 10.1 compatibility matrix lists the card on Linux and Windows 11. The limit is the usual one for AMD: llama.cpp and Ollama run on it through ROCm or Vulkan, while fine-tuning and much image and video tooling are written for CUDA first.
What it is good at
- 640 GB/s on 16 GB: about 1.4 times an RTX 5060 Ti 16 GB, at the same capacity.
- RDNA 4 runs FP8, which the RX 7000 series does not.
- AMD's ROCm 10.1 matrix lists it for Ubuntu, RHEL and Windows 11, and Ollama lists it for ROCm on Linux.
What it is not
- Sixteen gigabytes is still sixteen: the 32B class needs a 24 GB card.
- No CUDA. Tools that only ship CUDA builds, which is most fine-tuning and much of the image and video ecosystem, need a ROCm port or do not run.
- Ollama's ROCm list for Windows does not include it, so on Windows Ollama runs it through Vulkan.
The thing people get wrong: The RX 9070 without the XT has the same 16 GB, and AMD prints the same 640 GB/s for it, at 220 W. By the bandwidth formula the two cards share a speed ceiling for text generation; the XT's extra compute shows in prompt processing and image generation, not in chat speed.
Compared with the alternatives
- Radeon AI PRO R9700: 32 GB · 640 GB/s · best fit Muse Glimmer 30B at ~26 tok/s. AMD's 32 GB workstation card: RX 9070 XT bandwidth with twice the memory, at 300 W.
- RTX 5060 Ti: 16 GB · 448 GB/s · best fit Gemma 4 12B at ~45 tok/s. The newest 16 GB GeForce below the high end, and a clear step up from the 4060 Ti it replaces.
- RX 7900 XTX: 24 GB · 960 GB/s · best fit Muse Glimmer 30B at ~39 tok/s. AMD's 24 GB card: 3090-class memory and bandwidth, without CUDA.
All 39 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.
Questions
- Can an RX 9070 XT run a 70B model?
- Not at Q4_K_M in one card. Llama 3.3 70B needs about 45.8 GB at 8k context and this card can address 16 GB. Your options are a smaller model, a harsher quantisation with very little context, splitting across two cards, or renting a larger one for the hours you need it.
- What is the best model to run on an RX 9070 XT?
- Gemma 4 12B. At Q4_K_M and 8k context it needs about 8.2 GB of the 16 GB available and generates an estimated 64 tokens per second, which is faster than you can read. It also fits at Q8, at about 14.2 GB, which is worth taking when it fits.
- How many tokens per second does an RX 9070 XT generate?
- It depends entirely on the size of the model, because generating a token means reading every active weight out of memory. At 640 GB/s and 70% efficiency, this card produces an estimated 97 tokens per second on an 8B model at Q4, and about 215 on the largest model it holds, gpt-oss 20B. Reading speed is roughly 5 to 8 words per second, so anything above 25 feels immediate. Treat 97 as a ceiling on 640 GB/s rather than a measurement; the higher a figure, the further real runtimes fall below it.
- Is 16 GB enough for running LLMs locally?
- It runs 12 of the 34 models tracked here at Q4_K_M with 8k of context, up to 21B parameters. The honest test is not the model list but the context: Gemma 4 12B on this card holds about 131,072 tokens before memory runs out.
- RX 9070 XT or Radeon AI PRO R9700 for local models?
- The Radeon AI PRO R9700 holds more: 32 GB against 16 GB, so it runs 28 of these models to this card's 12. On speed, they are identical, and bandwidth is what sets chat speed.
- Is the RX 9070 XT good for AI?
- For running language models in llama.cpp or Ollama, yes: 16 GB at 640 GB/s gives a higher speed ceiling than an RTX 5060 Ti 16 GB on any model both can hold, and AMD's ROCm matrix lists the card on Linux and Windows. Fine-tuning and much image and video work still assume CUDA, so check each tool for ROCm support before you buy.
- Does the RX 9070 XT work with ROCm and Ollama on Windows?
- AMD's ROCm 10.1 compatibility matrix lists it on Windows 11, and llama.cpp's Vulkan backend runs on it. Ollama is the exception: its hardware page lists the RX 9070 XT for ROCm on Linux only, so on Windows Ollama uses its Vulkan backend, which it turns on by default.
- Memory: weights (parameters × bytes per parameter) + KV cache (2 × layers × KV heads × head dim × context × bytes for a standard transformer; sliding-window, hybrid and latent-attention models counted as they actually cache) + 0.5 GB runtime + 4% of weights. Full derivation in the VRAM calculator.
- Speed: 70% × bandwidth ÷ active weight bytes, single stream, no batching, prompt processing excluded. A ceiling, not a measurement: the 3 of 12 figures marked ≤ are where real runtimes fall furthest below the number, because so few weights are read per token. See the tokens-per-second estimator.
- Card specification (manufacturer figures): 16 GB, 640 GB/s, 304 W; see AMD's own page for this card.
- Model architecture values come from each model's published config; the self-hosted LLM guide explains what each one changes.