On September 1, 2026, NVIDIA and CrowdStrike published evaluation results for an agentic attack-defense system that connects offensive and defensive activities into a continuous closed loop operating at machine speed. The system was tested in an isolated environment modeled on NVIDIA accelerated computing infrastructure, using CrowdStrike Falcon telemetry and specialized agent harnesses built around NVIDIA Nemotron open models.
The defensive harness combined six mechanisms — schema knowledge base, telemetry grounding, specialized detection authoring with a customized Nemotron 3 Super, artifact linting, detection replay, and independent review — to produce grounded, testable detections. In the evaluated configuration, Nemotron 3 Ultra handled defensive orchestration while a post-trained Nemotron 3 Super served as a bounded expert for detection generation and repair, with training using NVIDIA NeMo Gym and NVIDIA NeMo RL for reinforcement learning with verifiable rewards.
What's new
- Closed-loop architecture: Red-agent harness (Red Tempest) executes attacks in an isolated representative environment; Blue-agent harness (Blue Solano) processes telemetry, generates detections, validates them, and deploys to the detection engine. The loop repeats until no further viable attack path remains.
- Model roles: Nemotron 3 Ultra for orchestration (reconstructing attack sequences, planning steps, invoking tools); fine-tuned Nemotron 3 Super as a bounded expert for detection authoring and repair.
- Specialized training: Nemotron 3 Super continually pretrained on cybersecurity knowledge, supervised fine-tuned on 9,349 detection-generation and multistep repair examples spanning 59 programmatically generated error types, then reinforced via NeMo Gym (query validation and execution in Falcon LogScale) and NeMo RL (group relative policy optimization with F1 overlap reward).
- Validation harness: Six mechanisms encode manual detection-engineering practices: schema tool prevents invented fields; telemetry grounding anchors to observed events; artifact linting rejects environment-specific strings; detection replay backtests against captured telemetry; independent judge reviews behavioral alignment and robustness.
- Environment: Sanitized natural-language specification of NVIDIA accelerated computing infrastructure translated into an isolated target cyber agent environment instrumented with Falcon sensors. NVIDIA security experts reviewed the environment and threat paths for realism.
Results
Backtesting across six independently seeded authoring sessions showed the optimized open-model pipeline achieved a 41.9% mean detection rate, a 2.5x improvement over the default harness with Nemotron 3 Ultra alone. Live-fire testing against eight unseen attacks from the same scenario family demonstrated that 45% of open-model detections generalized, compared to 29% for the frontier proprietary system tested. Three open-model detections qualified as "gold" (covering all eight attacks), while no frontier detections achieved gold status.
CrowdStrike separately reports that its Blue Solano defensive model is 13% more accurate than the leading proprietary frontier model tested, at 97% lower cost, in CrowdStrike internal evaluations.
Why it matters
The evaluation demonstrates that open models, when paired with specialized harnesses and reinforcement learning grounded in verifiable execution results, can match or exceed proprietary frontier models on detection generalization — at substantially lower cost. The closed-loop design shifts the bottleneck from manual handoffs between red and blue teams to an automated cycle that can iterate at machine speed, potentially allowing organizations to continuously test and improve coverage against evolving attack behaviors in their own environments.
Our take
The results suggest that the competitive moat in agentic cybersecurity may lie less in the base model and more in the harness architecture, validation tooling, and domain-specific RL pipelines that turn raw model capability into reliable, deployable detections. Organizations evaluating agentic security should weigh the engineering investment in those harnesses against the recurring cost of frontier-model APIs.
Series: 1. NVIDIA Releases NemotronLabs VoiceChat 11B, an Open Full-Duplex Speech Model with Live Tool Calling · 2. NVIDIA Nemotron 3.5 Lightning targets agent execution layer with 30B MoE model · 3. NVIDIA NeMo Switchyard Routes AI Agent Workloads Across Models to Cut Cost and Latency · 4. NVIDIA BioNeMo Agent Toolkit integrates with Claude Science for protein structure prediction · 5. NVIDIA and CrowdStrike validate closed-loop agentic cybersecurity system with Nemotron models · NVIDIA Nemotron & NeMo
