Flint 40B-A2.5B (MoE)
Flint 40B-A2.5B (MoE) needs 24.8 GB of VRAM at Q4_K_M with 8k context, computed from its config.json as checked on 3 November 2026. The smallest card that holds it has 32 GB, and the fastest desktop card that runs it, the RTX 5090, generates at most 865 tokens per second.
- Parameters
- 39.8B, 2.5B active
- Context
- 262,144 tokens
- Licence
- Apache 2.0
- Released
- October 2026
- Config checked
- 3 November 2026
Open Flint 40B-A2.5B (MoE) in the LLM checker
Which card size runs it
24.8 GB against each memory size. A model may use 95% of a card, and only what is left over, or missing, is coloured; tight means above 85%, with little room for a longer conversation.
Other quantisations, context lengths or a Q8 KV cache: the VRAM calculator.
Computed 3 November 2026Which GPU to pick
The smallest card here that loads it at Q4_K_M with 8k of context is the RTX 5090 (32 GB): 24.8 GB needed, an estimated 865 tokens per second. The same card still has room at 32,768 tokens of context (25.5 GB), so there is no reason to buy above it for this model.
At Q8_0, which is near-lossless, the card is the RTX A6000 (48 GB), needing 44.6 GB. On Apple silicon, a Mac mini M4 Pro holds it with 32,768 tokens of context at an estimated 132 tokens per second; prompt processing is slower than on Nvidia, which long documents make obvious. Among unified-memory desktops, the Ryzen AI Max+ 395 holds it at 32,768 tokens in 25.5 GB, at an estimated 124 tokens per second.
Speed on each card
| GPU | Fits | Est. tokens/s | Feels like | Memory | Bandwidth |
|---|---|---|---|---|---|
| H100 SXM | Yes | ≤ 1,617 | faster than you can read | 80 GB | 3,350 GB/s |
| A100 | Yes | ≤ 984 | faster than you can read | 80 GB | 2,039 GB/s |
| RTX 5090 | Yes | ≤ 865 | faster than you can read | 32 GB | 1,792 GB/s |
| RTX PRO 6000 Blackwell | Yes | ≤ 865 | faster than you can read | 96 GB | 1,792 GB/s |
| RTX PRO 5000 Blackwell | Yes | ≤ 649 | faster than you can read | 48 GB | 1,344 GB/s |
| RTX 6000 Ada | Yes | ≤ 463 | faster than you can read | 48 GB | 960 GB/s |
| RTX PRO 4500 Blackwell | Yes | ≤ 433 | faster than you can read | 32 GB | 896 GB/s |
| L40S | Yes | ≤ 417 | faster than you can read | 48 GB | 864 GB/s |
| RTX A6000 | Yes | ≤ 371 | faster than you can read | 48 GB | 768 GB/s |
| Radeon AI PRO R9700 | Yes | ≤ 309 | faster than you can read | 32 GB | 640 GB/s |
| Arc Pro B70 | Yes | ≤ 294 | faster than you can read | 32 GB | 608 GB/s |
| RTX Spark | Yes | ≤ 145 | faster than you can read | 128 GB (102 for the GPU) | 300 GB/s |
| DGX Spark | Yes | ≤ 132 | faster than you can read | 128 GB (126 for the GPU) | 273 GB/s |
| Ryzen AI Max+ 395 | Yes | ≤ 124 | faster than you can read | 128 GB (96 for the GPU) | 256 GB/s |
| RTX 3090 | No | - | does not fit | 24 GB | 936 GB/s |
| RTX 4090 | No | - | does not fit | 24 GB | 1,008 GB/s |
| RX 7900 XTX | No | - | does not fit | 24 GB | 960 GB/s |
| RTX PRO 4000 Blackwell | No | - | does not fit | 24 GB | 672 GB/s |
| RTX PRO 4000 Blackwell SFF | No | - | does not fit | 24 GB | 432 GB/s |
| Arc Pro B60 | No | - | does not fit | 24 GB | 456 GB/s |
| RTX 5090 Laptop GPU | No | - | does not fit | 24 GB | 896 GB/s |
| RX 7900 XT | No | - | does not fit | 20 GB | 800 GB/s |
| RTX 4060 Ti | No | - | does not fit | 16 GB | 288 GB/s |
| RTX 5060 Ti | No | - | does not fit | 16 GB | 448 GB/s |
| RTX 5080 | No | - | does not fit | 16 GB | 960 GB/s |
| RTX 5070 Ti | No | - | does not fit | 16 GB | 896 GB/s |
| RTX 4080 SUPER | No | - | does not fit | 16 GB | 736 GB/s |
| RX 9070 XT | No | - | does not fit | 16 GB | 640 GB/s |
| RX 9060 XT | No | - | does not fit | 16 GB | 320 GB/s |
| RTX 3060 | No | - | does not fit | 12 GB | 360 GB/s |
| RTX 4070 | No | - | does not fit | 12 GB | 504 GB/s |
| RTX 5070 | No | - | does not fit | 12 GB | 672 GB/s |
| Arc B580 | No | - | does not fit | 12 GB | 456 GB/s |
Single-stream decode, from memory bandwidth at 70% efficiency. Prompt processing and batching not included. These are ceilings, not measurements: Flint 40B-A2.5B (MoE) reads only 1.5 GB of weights per token at Q4, so little that the per-token costs the formula leaves out decide much of the real speed. The 14 figures marked ≤ are where a real runtime falls furthest below the number shown. See the speed estimator for other quantisations and Apple hardware, or every card compared if you are choosing hardware rather than a model.
Computed 3 November 2026Memory by quantisation and context
| Quant | 4,096 ctx | 8,192 ctx | 32,768 ctx |
|---|---|---|---|
| Q8_0 | 44.5 GB · fits 48 GB | 44.6 GB · fits 48 GB | 45.4 GB · fits 48 GB |
| Q5_K_M | 30 GB · fits 32 GB | 30.2 GB · fits 32 GB | 30.9 GB · fits 48 GB |
| Q4_K_M | 24.7 GB · fits 32 GB | 24.8 GB · fits 32 GB | 25.5 GB · fits 32 GB |
Weights at this quant: 23.1 GB at Q4. Every extra 1,000 tokens of context adds about 0.029 GB of KV cache at FP16, on top of a fixed 0.037 GB that the sliding-window layers hold once their window is full. Try other settings in the VRAM calculator.
About the model
Flint 40B-A2.5B (MoE) has 39.8B parameters with 2.5B active per token (mixture of experts). It has 49 layers: 35 use sliding-window attention over the current token and the 512 before it, and 14 are full attention, all with 4 KV heads of dimension 128. Recursive's coding-agent model from October 2026, trained from scratch: 39.8B of weights across 256 experts, with about 2.5B read per token. Recursive built it to investigate codebases, fix bugs, implement features and write tests, and prepares quantised builds for NVIDIA and Apple silicon.
Text only 262,144-token window Apache 2.0 October 2026 2.5B of 39.8B active
Reasoning. Reasons before it answers. The card asks you to keep the returned reasoning in the conversation history and to allow at least 32,768 output tokens for hard problems.
All 256 experts stay in memory; about 2.5B are active per token, the output head included. A runtime without a sliding-window cache stores every layer at full length and needs far more than this at long context.
This is a coding model, tuned for writing and editing code and for agentic tools that read whole repositories, which is where a long context earns its memory.
Native context window: 262,144 tokens. Licence: Apache 2.0, a permissive licence. Weights published in October 2026.
Good at
- A coding agent at small-model speed: about 2.5B parameters are read per token.
- Long sessions: 35 of its 49 layers only attend to the last 513 tokens, so the cache grows in 14.
- A 32 GB card such as the RTX 5090, where it fits at Q4_K_M with a long context.
- GGUF builds from unsloth, LM Studio and ggml-org, and Apache 2.0.
Watch out for
- It is a specialist for coding agents, not a general chat model.
- A new architecture: Recursive ships no Transformers code yet and serves it through its own SGLang kit, so check that your local runtime supports Flint before downloading.
- On a 24 GB card it does not fit at Q4_K_M; Q3_K_M fits, at some cost in quality.
- Long reasoning can use up the output budget before the answer is complete, the card warns.
- Recursive is a new lab with no track record yet. Try it on your own code before relying on it.
Similar models
The models that need about the same memory as Flint 40B-A2.5B (MoE) at Q4_K_M and 8k, which makes them the real alternatives on whatever card you have. The context column is what each extra 1,000 tokens costs, and it is where models of the same size differ most.
| Model | Size | Needs | Per 1k context | Window | Released |
|---|---|---|---|---|---|
| Flint 40B-A2.5B (MoE) | 39.8B · 2.5B active | 24.8 GB | 0.029 GB | 256k | 2026-10 |
| Sage-A1.5 35B (MoE) | 36B · 3B active | 22.4 GB | 0.02 GB | 256k | 2026-10 |
| Qwen3.6 35B-A3B (MoE) | 36B · 3B active | 22.4 GB | 0.02 GB | 256k | 2026-04 |
| Qwen3 32B | 32.8B | 22.4 GB | 0.262 GB | 40k | 2025-04 |
| DeepSeek-R1 Distill Qwen 32B | 32.8B | 22.4 GB | 0.262 GB | 128k | 2025-01 |
See every model compared, which models fit on 8 to 96 GB, or the cheapest hour of every cloud card.
Questions
- Can I run Flint 40B-A2.5B (MoE) on a 24 GB card?
- Not comfortably. At Q4_K_M and 8k context it needs about 24.8 GB. The smallest tier that fits is 32 GB.
- How much VRAM does Flint 40B-A2.5B (MoE) need at Q8?
- About 44.6 GB at 8k context, or 45.4 GB at 32k. Q8 is near-lossless; use it when it fits.
- What is the context length of Flint 40B-A2.5B (MoE)?
- 262,144 tokens natively. Holding all of it at Q4_K_M takes about 32.1 GB, of which 7.6 GB is context.
- Why does Flint 40B-A2.5B (MoE) need so little memory for long context?
- Because most of its layers do not keep a cache that grows. It has 49 layers: 35 use sliding-window attention over the current token and the 512 before it, and 14 are full attention, all with 4 KV heads of dimension 128. At 32k tokens the context costs about 1 GB; if all 49 layers kept a conventional full-length cache, the same conversation would cost about 3.3 GB. That saving depends on the runtime implementing the sliding-window cache. llama.cpp and vLLM do; a build that does not will use far more.
- Flint or Qwen3-Coder-Next?
- Both are mixture-of-experts models for coding agents that read about 2.5B to 3B parameters per token. Flint holds 39.8B of weights and fits a 32 GB card at Q4_K_M; Qwen3-Coder-Next holds 79.7B and needs an 80 GB card or a large-memory Mac. Flint reasons before it answers; Qwen3-Coder-Next answers directly, with no reasoning blocks. The tables on both pages show the figures.
- How fast is Flint 40B-A2.5B (MoE) on an RTX 4090?
- It does not fit a 4090 at Q4 and 8k context, so speed would collapse to CPU offloading. Use a larger card or a smaller quantisation.
- Architecture values from config.json in RecursiveSI/flint on Hugging Face, and its model card, checked 3 November 2026. Every figure on this page is computed from those values; check them before buying hardware for this model.
- Estimates assume a single conversation on a card that is otherwise free. A desktop on the same GPU takes 0.5 to 2 GB.