NVIDIA has released Nemotron 3.5 Lightning, an open 30-billion-parameter mixture-of-experts model that activates only 3 billion parameters per token. The model is purpose-built for the high-volume execution layer of long-running AI agents. It handles tool calls, result validation, and subagent delegation. Larger frontier models such as Nemotron 3 Ultra take on planning and orchestration.
The release includes BF16 and NVFP4 quantized checkpoints. It also ships with speculative decoding draft models called DSpark and DFlash. Permissive OpenMDW-1.1 licensing covers weights, training data, and recipes. NVIDIA also introduced NeMo Switchyard, a routing library that directs each workflow step to the most suitable model in a developer's pool.
What's new
- Architecture: 30B total parameters, 3B active via sparse MoE routing
- Precision formats: BF16 and NVFP4 (specialized kernels for Blackwell, Hopper, Ampere GPUs)
- Speculative decoding: Native multi-token prediction baked into pretraining; DSpark draft model for DGX Spark and low-concurrency data center workloads; DFlash draft model for measurement against other drafts
- Context window: Up to 1 million tokens
- Customization: LoRA or full SFT via NeMo Automodel and NeMo Megatron Bridge; RL and environment-based evaluations with NeMo RL and NeMo Gym
- Open dataset: Nemotron-RL Agentic Terminal Pivot released alongside the model
- Deployment targets: DGX Spark, GeForce RTX 5090, Jetson, cloud GPUs; supported by vLLM, SGLang, TensorRT-LLM, Ollama, LM Studio, Unsloth, Exo
- Cloud availability: Amazon SageMaker JumpStart (model IDs: huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-nvfp4 and huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-bf16); Google Cloud Gemini Enterprise Agent Platform; Microsoft Foundry; OCI Enterprise AI
- Licensing: OpenMDW-1.1 (permissive)
On the Artificial Analysis Intelligence Index, Nemotron 3.5 Lightning sits on the accuracy-speed Pareto frontier for small open models. On PinchBench, it reaches 86% accuracy while completing 10,000 tasks 30% faster than Qwen3 35B at comparable accuracy. NVIDIA reports up to 4x higher output throughput versus similar-sized dense models.
The AWS SageMaker JumpStart blog adds benchmark detail for the quantized variant: MMLU Pro 81.62 (NVFP4) vs 81.94 (BF16), GPQA Diamond 75.57 vs 75.44, SWE-bench Verified 52.80 vs 51.56, PinchBench 83.43 vs 85.37, IFBench 72.88 vs 71.88, AA-LCR 49.19 vs 52.00. NVFP4 remains close to BF16 across most tasks.
Why it matters
Always-on agents spend the majority of their token budget on repetitive execution steps — git pulls, API calls, output formatting — not on high-level reasoning. Running a frontier model for every step inflates cost and latency. Nemotron 3.5 Lightning formalizes a system-of-models approach: a small, fast MoE model handles the execution volume, while larger models are reserved for planning. NeMo Switchyard makes that division of labor operational by routing each request to the appropriate model.
The 1-million-token context window lets agents carry accumulated state across long sessions without repeated re-grounding. Local deployment on DGX Spark and RTX 5090 brings capable agentic execution to desktop and edge scenarios, reducing cloud dependency for latency-sensitive or privacy-conscious workloads.
Our take
The benchmarks are promising, but the real test is whether developers adopt the routing discipline Switchyard requires. Most agent frameworks today still default to a single large model; splitting workloads across a model fleet adds operational complexity that only pays off at sustained high volume.
