Gemma 4 31B has 31.3B parameters. It has 60 layers: 50 use sliding-window attention over the last 1,024 tokens (16 KV heads of dimension 256), and 10 are global layers with 4 KV heads of dimension 512. At the everyday setting, Q4_K_M and 8k context, it needs an estimated 20.9 GB, so the smallest card that fits is 24 GB.
What Gemma 4 31B is
The dense flagship of Google DeepMind's Gemma 4 family, spring 2026. Vision-language, a 256k window, configurable thinking, and an Apache licence in place of Gemma 3's custom terms.
Text and images 262,144-token window Apache 2.0 March 2026
Reasoning. Configurable thinking modes, per the model card.
Memory by quantisation and context
| Quant | 4,096 ctx | 8,192 ctx | 32,768 ctx |
|---|---|---|---|
| Q8_0 | 36.2 GB · fits 48 GB | 36.5 GB · fits 48 GB | 38.5 GB · fits 48 GB |
| Q5_K_M | 24.8 GB · fits 32 GB | 25.1 GB · fits 32 GB | 27.1 GB · fits 32 GB |
| Q4_K_M | 20.6 GB · fits 24 GB | 20.9 GB · fits 24 GB | 22.9 GB · fits 32 GB |
Weights at this quant: 18.2 GB at Q4. Every extra 1,000 tokens of context adds about 0.082 GB of KV cache at FP16, on top of a fixed 0.839 GB that the sliding-window layers hold once their window is full. Try other settings in the VRAM calculator.
Which GPU to pick for Gemma 4 31B
The smallest card here that loads it at Q4_K_M with 8k of context is the RTX 3090 (24 GB): 20.9 GB needed, an estimated 36 tokens per second, and a tight fit that leaves little for the conversation. For room to work, meaning 32,768 tokens of context while staying under 85% of memory, step up to the RTX 5090 (32 GB), which needs 22.9 GB for that and generates at about 69 tokens per second.
At Q8_0, which is near-lossless, the card is the L40S (48 GB), needing 36.5 GB. On Apple silicon, a MacBook Pro M4 Max holds it with 32,768 tokens of context at an estimated 21 tokens per second; prompt processing is slower than on Nvidia, which long documents make obvious. Among unified-memory desktops, the Ryzen AI Max+ 395 holds it at 32,768 tokens in 22.9 GB, at an estimated 10 tokens per second. To check a card that is not on this page, or a different context length, use the can-I-run-it checker, which opens on this model.
Speed by GPU, at Q4 and 8k context
| GPU | Memory | Bandwidth | Fits | Est. tokens/s | Feels like |
|---|---|---|---|---|---|
| RTX 3060 12 GB | 12 GB | 360 GB/s | no | – | does not fit |
| RTX 4060 Ti 16 GB | 16 GB | 288 GB/s | no | – | does not fit |
| RTX 4070 12 GB | 12 GB | 504 GB/s | no | – | does not fit |
| RTX 3090 24 GB | 24 GB | 936 GB/s | tight | 36 | comfortable for chat |
| RTX 4090 24 GB | 24 GB | 1,008 GB/s | tight | 39 | comfortable for chat |
| RTX 5090 32 GB | 32 GB | 1,792 GB/s | yes | 69 | faster than you can read |
| RTX 6000 Ada 48 GB | 48 GB | 960 GB/s | yes | 37 | comfortable for chat |
| L40S 48 GB | 48 GB | 864 GB/s | yes | 33 | comfortable for chat |
| RTX PRO 6000 Blackwell 96 GB | 96 GB | 1,792 GB/s | yes | 69 | faster than you can read |
| A100 80 GB | 80 GB | 2,039 GB/s | yes | 79 | faster than you can read |
| H100 SXM 80 GB | 80 GB | 3,350 GB/s | yes | ≤ 129 | faster than you can read |
| RTX 5060 Ti 16 GB | 16 GB | 448 GB/s | no | – | does not fit |
| Radeon RX 7900 XTX 24 GB | 24 GB | 960 GB/s | tight | 37 | comfortable for chat |
| DGX Spark 128 GB (unified) | 128 GB (126 for the GPU) | 273 GB/s | yes | 11 | slow; fine for batch jobs |
| Ryzen AI Max+ 395 128 GB (unified) | 128 GB (96 for the GPU) | 256 GB/s | yes | 10 | slow; fine for batch jobs |
Single-stream decode, from memory bandwidth at 70% efficiency. Prompt processing and batching not included. These are ceilings, not measurements: Gemma 4 31B reads only 18.2 GB of weights per token at Q4, so little that the per-token costs the formula leaves out decide much of the real speed. The figure marked ≤ is where a real runtime falls furthest below the number shown. See the speed estimator for other quantisations and Apple hardware, or every card compared if you are choosing hardware rather than a model.
What Gemma 4 31B is good at, and what it is not
Good at
- The hardest work a 24 GB card can take on: a dense 31B that still fits one at Q4.
- Long context at moderate cost: 50 of its 60 layers are sliding-window.
- Commercial use under Apache 2.0.
Watch out for
- It fills a 24 GB card. It loads and runs, but a 32 GB card is where it has room for a long document.
- Its global layers are wider than Gemma 3's, so per token it costs more context memory than Qwen3.8 27B does.
- No audio input; in Gemma 4 that is the 12B.
Similar models
The models that need about the same memory as Gemma 4 31B at Q4_K_M and 8k, which makes them the real alternatives on whatever card you have. The context column is what each extra 1,000 tokens costs, and it is where models of the same size differ most.
| Model | Size | Needs | Per 1k context | Window | Released |
|---|---|---|---|---|---|
| Gemma 4 31B | 31.3B | 20.9 GB | 0.082 GB | 256k | 2026-03 |
| GLM-4.7 Flash 30B-A3B (MoE) | 31.2B · 3B active | 19.8 GB | 0.054 GB | 198k | 2026-01 |
| Qwen3 30B-A3B (MoE) | 30.5B · 3.3B active | 19.7 GB | 0.098 GB | 40k | 2025-04 |
| Nemotron 3.5 Lightning 30B-A3B (MoE) | 31.6B · 3B active | 19.7 GB | 0.006 GB | 256k | 2026-08 |
| Qwen3 32B | 32.8B | 22.4 GB | 0.262 GB | 40k | 2025-04 |
Notes
- A runtime without a sliding-window cache stores every layer at full length and needs far more than this at long context. The parameter count includes the vision encoder; a text-only GGUF is slightly smaller.
- Native context window: 262,144 tokens. Licence: Apache 2.0, a permissive licence.
- Weights published in March 2026. It follows Gemma 3 27B.
- Architecture values from config.json in google/gemma-4-31B-it on Hugging Face. Every figure on this page is computed from those values; check them before buying hardware for this model.
- Estimates assume a single conversation on a card that is otherwise free. A desktop on the same GPU takes 0.5 to 2 GB.
Questions
Can I run Gemma 4 31B on a 24 GB card?
Yes, at Q4_K_M and 8k context it needs about 20.9 GB, which fits a 24 GB card tightly.
How much VRAM does Gemma 4 31B need at Q8?
About 36.5 GB at 8k context, or 38.5 GB at 32k. Q8 is near-lossless; use it when it fits.
What is the context length of Gemma 4 31B?
262,144 tokens natively. Holding all of it at Q4_K_M takes about 41.7 GB, of which 22.3 GB is context.
Why does Gemma 4 31B need so little memory for long context?
Because most of its layers do not keep a cache that grows. It has 60 layers: 50 use sliding-window attention over the last 1,024 tokens (16 KV heads of dimension 256), and 10 are global layers with 4 KV heads of dimension 512. At 32k tokens the context costs about 3.5 GB; if all 60 layers kept a conventional full-length cache, the same conversation would cost about 32.2 GB. That saving depends on the runtime implementing the sliding-window cache. llama.cpp and vLLM do; a build that does not will use far more.
How fast is Gemma 4 31B on an RTX 4090?
At most about 39 tokens per second at Q4, single stream, which is comfortable for chat. That is a ceiling worked out from the card's memory bandwidth, not a measurement.
See every model compared and which models fit on 8 to 96 GB, or run it without buying the card: Nodegrove attaches a 24, 48 or 96 GB GPU to a workspace that stays saved.