Gemma 4 26B-A4B (MoE): VRAM requirements and which GPUs run it

Gemma 4 26B-A4B (MoE) has 25.8B parameters with 3.8B active per token (mixture of experts). It has 30 layers: 25 use sliding-window attention over the last 1,024 tokens (8 KV heads of dimension 256), and 5 are global layers with 2 KV heads of dimension 512. At the everyday setting, Q4_K_M and 8k context, it needs an estimated 16.4 GB, so the smallest card that fits is 24 GB.

What Gemma 4 26B-A4B (MoE) is

The mixture-of-experts member of Gemma 4, spring 2026: 128 experts with 8 active plus one shared, 3.8B parameters read per token. Google positions it as the fast alternative to the dense 31B.

Text and images 262,144-token window Apache 2.0 March 2026 3.8B of 25.8B active

Reasoning. Configurable thinking modes, per the model card.

Memory by quantisation and context

Weights at this quant: 15 GB at Q4. Every extra 1,000 tokens of context adds about 0.02 GB of KV cache at FP16, on top of a fixed 0.21 GB that the sliding-window layers hold once their window is full. Try other settings in the VRAM calculator.

Which GPU to pick for Gemma 4 26B-A4B (MoE)

The smallest card here that loads it at Q4_K_M with 8k of context is the RTX 3090 (24 GB): 16.4 GB needed, an estimated 297 tokens per second. The same card still has room at 32,768 tokens of context (16.9 GB), so there is no reason to buy above it for this model.

At Q8_0, which is near-lossless, the card is the RTX 5090 (32 GB), needing 29.3 GB. On Apple silicon, a MacBook Pro M4 Max holds it with 32,768 tokens of context at an estimated 173 tokens per second; prompt processing is slower than on Nvidia, which long documents make obvious. Among unified-memory desktops, the Ryzen AI Max+ 395 holds it at 32,768 tokens in 16.9 GB, at an estimated 81 tokens per second. To check a card that is not on this page, or a different context length, use the can-I-run-it checker, which opens on this model.

Speed by GPU, at Q4 and 8k context

Single-stream decode, from memory bandwidth at 70% efficiency. Prompt processing and batching not included. These are ceilings, not measurements: Gemma 4 26B-A4B (MoE) reads only 2.2 GB of weights per token at Q4, so little that the per-token costs the formula leaves out decide much of the real speed. The 9 figures marked ≤ are where a real runtime falls furthest below the number shown. See the speed estimator for other quantisations and Apple hardware, or every card compared if you are choosing hardware rather than a model.

What Gemma 4 26B-A4B (MoE) is good at, and what it is not

Good at

• Speed on a 24 GB card: it answers like a 4B model while holding 26B of weights.
• Long context cheaply: 25 of its 30 layers are sliding-window.
• Apache 2.0.

Watch out for

• All 25.8B parameters must fit in memory even though few are used per token. It is fast, not small.
• A mixture-of-experts model this sparse trades depth for speed. For the hardest reasoning, the dense 31B is the one Google built for it.
• No audio input; in Gemma 4 that is the 12B and the E-series.

Similar models

The models that need about the same memory as Gemma 4 26B-A4B (MoE) at Q4_K_M and 8k, which makes them the real alternatives on whatever card you have. The context column is what each extra 1,000 tokens costs, and it is where models of the same size differ most.

Notes

• All 128 experts stay in memory; 3.8B are active per token. The parameter count includes the vision encoder; a text-only GGUF is slightly smaller.
• Native context window: 262,144 tokens. Licence: Apache 2.0, a permissive licence.
• Weights published in March 2026.
• Architecture values from config.json in google/gemma-4-26B-A4B-it on Hugging Face, and its model card. Every figure on this page is computed from those values; check them before buying hardware for this model.
• Estimates assume a single conversation on a card that is otherwise free. A desktop on the same GPU takes 0.5 to 2 GB.

Questions

Can I run Gemma 4 26B-A4B (MoE) on a 24 GB card?

Yes, at Q4_K_M and 8k context it needs about 16.4 GB, which fits a 24 GB card with room.

How much VRAM does Gemma 4 26B-A4B (MoE) need at Q8?

About 29.3 GB at 8k context, or 29.8 GB at 32k. Q8 is near-lossless; use it when it fits.

What is the context length of Gemma 4 26B-A4B (MoE)?

262,144 tokens natively. Holding all of it at Q4_K_M takes about 21.6 GB, of which 5.6 GB is context.

Why does Gemma 4 26B-A4B (MoE) need so little memory for long context?

Because most of its layers do not keep a cache that grows. It has 30 layers: 25 use sliding-window attention over the last 1,024 tokens (8 KV heads of dimension 256), and 5 are global layers with 2 KV heads of dimension 512. At 32k tokens the context costs about 0.9 GB; if all 30 layers kept a conventional full-length cache, the same conversation would cost about 8.1 GB. That saving depends on the runtime implementing the sliding-window cache. llama.cpp and vLLM do; a build that does not will use far more.

How fast is Gemma 4 26B-A4B (MoE) on an RTX 4090?

At most about 320 tokens per second at Q4, single stream, which is faster than you can read. That is a ceiling worked out from the card's memory bandwidth, not a measurement, and at a figure this high real runtimes land well below it, because the costs the formula leaves out take a growing share of each token.

See every model compared and which models fit on 8 to 96 GB, or run it without buying the card: Nodegrove attaches a 24, 48 or 96 GB GPU to a workspace that stays saved.