Nemotron 3.5 Lightning 30B-A3B (MoE): VRAM requirements and which GPUs run it

Nemotron 3.5 Lightning 30B-A3B (MoE) has 31.6B parameters with 3B active per token (mixture of experts). It has 52 blocks, of which only 6 are attention layers (2 KV heads of dimension 128); 23 are Mamba-2 layers with a small fixed state and 23 are mixture-of-experts layers with no cache at all. At the everyday setting, Q4_K_M and 8k context, it needs an estimated 19.7 GB, so the smallest card that fits is 24 GB.

What Nemotron 3.5 Lightning 30B-A3B (MoE) is

NVIDIA's open 30B-A3B model, August 2026, released with its training data and recipes. A three-way hybrid: Mamba-2 layers, mixture-of-experts layers, and only six attention layers, which makes it the cheapest model here to give a long context.

Text only 262,144-token window OpenMDW 1.1 August 2026 3B of 31.6B active

Reasoning. A reasoning and chat model; reasoning output is parsed separately by the runtime.

Memory by quantisation and context

Weights at this quant: 18.3 GB at Q4. Every extra 1,000 tokens of context adds about 0.006 GB of KV cache at FP16, on top of a fixed 0.05 GB of recurrent state that does not grow. Try other settings in the VRAM calculator.

Which GPU to pick for Nemotron 3.5 Lightning 30B-A3B (MoE)

The smallest card here that loads it at Q4_K_M with 8k of context is the RTX 3090 (24 GB): 19.7 GB needed, an estimated 377 tokens per second. The same card still has room at 32,768 tokens of context (19.8 GB), so there is no reason to buy above it for this model.

At Q8_0, which is near-lossless, the card is the L40S (48 GB), needing 35.4 GB. On Apple silicon, a MacBook Pro M4 Max holds it with 32,768 tokens of context at an estimated 220 tokens per second; prompt processing is slower than on Nvidia, which long documents make obvious. Among unified-memory desktops, the Ryzen AI Max+ 395 holds it at 32,768 tokens in 19.8 GB, at an estimated 103 tokens per second. To check a card that is not on this page, or a different context length, use the can-I-run-it checker, which opens on this model.

Speed by GPU, at Q4 and 8k context

Single-stream decode, from memory bandwidth at 70% efficiency. Prompt processing and batching not included. These are ceilings, not measurements: Nemotron 3.5 Lightning 30B-A3B (MoE) reads only 1.7 GB of weights per token at Q4, so little that the per-token costs the formula leaves out decide much of the real speed. The 11 figures marked ≤ are where a real runtime falls furthest below the number shown. See the speed estimator for other quantisations and Apple hardware, or every card compared if you are choosing hardware rather than a model.

What Nemotron 3.5 Lightning 30B-A3B (MoE) is good at, and what it is not

Good at

• Very long context on a 24 GB card: six attention layers with two KV heads each is almost no cache at all.
• Teams that want to see, and reuse, the training data and recipe, not only the weights.
• Fine-tuning: NVIDIA describes the BF16 release as the starting point for customisation.

Watch out for

• The BF16 checkpoint is meant for customisation. For running it, NVIDIA points to its NVFP4 and GGUF releases.
• OpenMDW 1.1 is its own licence; read it rather than assuming Apache.
• The card lists six supported languages: English, Spanish, French, German, Italian and Japanese.

Similar models

The models that need about the same memory as Nemotron 3.5 Lightning 30B-A3B (MoE) at Q4_K_M and 8k, which makes them the real alternatives on whatever card you have. The context column is what each extra 1,000 tokens costs, and it is where models of the same size differ most.

Notes

• Native context window: 262,144 tokens. Licence: OpenMDW 1.1, a custom licence with its own conditions; read it before commercial use.
• Weights published in August 2026.
• Architecture values from config.json in nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 on Hugging Face, and its model card. Every figure on this page is computed from those values; check them before buying hardware for this model.
• Estimates assume a single conversation on a card that is otherwise free. A desktop on the same GPU takes 0.5 to 2 GB.

Questions

Can I run Nemotron 3.5 Lightning 30B-A3B (MoE) on a 24 GB card?

Yes, at Q4_K_M and 8k context it needs about 19.7 GB, which fits a 24 GB card with room.

How much VRAM does Nemotron 3.5 Lightning 30B-A3B (MoE) need at Q8?

About 35.4 GB at 8k context, or 35.6 GB at 32k. Q8 is near-lossless; use it when it fits.

What is the context length of Nemotron 3.5 Lightning 30B-A3B (MoE)?

262,144 tokens natively. Holding all of it at Q4_K_M takes about 21.2 GB, of which 1.7 GB is context.

Why does Nemotron 3.5 Lightning 30B-A3B (MoE) need so little memory for long context?

Because most of its layers do not keep a cache that grows. It has 52 blocks, of which only 6 are attention layers (2 KV heads of dimension 128); 23 are Mamba-2 layers with a small fixed state and 23 are mixture-of-experts layers with no cache at all. At 32k tokens the context costs about 0.3 GB; if all 52 layers kept a conventional full-length cache, the same conversation would cost about 1.7 GB.

How fast is Nemotron 3.5 Lightning 30B-A3B (MoE) on an RTX 4090?

At most about 406 tokens per second at Q4, single stream, which is faster than you can read. That is a ceiling worked out from the card's memory bandwidth, not a measurement, and at a figure this high real runtimes land well below it, because the costs the formula leaves out take a growing share of each token.

See every model compared and which models fit on 8 to 96 GB, or run it without buying the card: Nodegrove attaches a 24, 48 or 96 GB GPU to a workspace that stays saved.