Google DeepMind has unveiled Gemini Robotics ER 2, a new embodied reasoning model designed to serve as a high-level brain for robots. Announced on July 30, 2026, the system introduces real-time video understanding, multi-step task orchestration, and native multi-robot collaboration — capabilities aimed at making robots more helpful and safer in physical environments. The model is now accessible to developers through the Gemini API, Google AI Studio, and in private preview on the Gemini Enterprise Agent Platform.
The release marks a significant upgrade over the previous Gemini Robotics ER 1.6. By processing continuous video feeds, robots can now track their own progress, adapt when something goes wrong, and determine precisely when to transition to the next step. Google DeepMind positions ER 2 as a physical agent that orchestrates low-level vision-language-action (VLA) models and robotics APIs while maintaining the sub-second latency required for safe real-world operation. The company also emphasizes that ER 2 can natively call external tools such as Google Search or any user-defined function, expanding the range of information a robot can retrieve during a task.
What's New: Video Understanding, Tool Orchestration, and Multi-Robot Collaboration
Gemini Robotics ER 2 centers on three core advances. First, temporal intelligence: the model performs continuous progress classification across five granular levels (0–20%, 20–40%, 40–60%, 60–80%, 80–100%) and achieves 57.4% accuracy on progress classification tasks. It also delivers precision moment-finding at 91.3% accuracy with a 0.96-second mean absolute distance, enabling robots to identify the exact frame where a critical event occurs — such as when to stop pouring liquid — and switch tasks without the "stop-and-think" pauses that plagued earlier systems. These metrics are reported on benchmarks that evaluate frame-level progress tracking and event localization in realistic manipulation sequences.
Second, tool orchestration has been overhauled. Developers can declare low-level control interfaces — VLA models, navigation APIs, or custom functions — as tools and stream multimodal video, audio, or text directly into ER 2. The model integrates with the Gemini Live API via a bidirectional streaming endpoint optimized for latency-sensitive tasks. In evaluations across three control modes (real VLA, simulated VLA, and human tele-operation), ER 2 consistently outperforms ER 1.6. A demonstration with Boston Dynamics' Spot robot shows ER 2 orchestrating Spot's navigation and manipulator APIs to fetch objects on natural language commands; the code is available on GitHub for developers to inspect and extend.
- Progress classification: 57.4% accuracy across five progress levels
- Moment-finding: 91.3% accuracy, 0.96s mean absolute distance, 4x execution speed vs. prior generation
- Tool orchestration: improved across real VLA, sim VLA, and human tele-op control modes
- Gemini Live API integration for sub-second bidirectional streaming
- Multi-robot collaboration enabling heterogeneous robots to share semantic understanding
Third, multi-robot collaboration allows diverse machines — such as Apptronik's Apollo 2 humanoid and Franka's F3 Duo arm — to communicate via a shared semantic understanding, handing off subtasks to complete complex workflows a single robot could not manage alone. This addresses the reality that no single form factor excels at every task: wheeled rovers may suit indoor navigation, while humanoids handle uneven terrain or bimanual manipulation. The collaboration layer is built on a common language of task descriptors that ER 2 generates, letting each robot execute its portion while the high-level model monitors overall progress.
Why It Matters: Deployment, Safety, and the Path to Generalist Robots
The practical significance of ER 2 lies in closing the gap between high-level reasoning and real-time physical execution. Historically, robotics stacks have suffered from a "think-act" disconnect: a planner reasons slowly, then hands off to a controller that executes blindly. ER 2's integration with the Gemini Live API means the model can "think" about what comes next while simultaneously commanding action models, yielding fluid orchestration without jarring pauses. This is critical for tasks requiring continuous feedback — tightening a bolt, tying a bag, or navigating around a moving person — where latency directly impacts safety and success rates.
Safety benchmarks show measurable gains. ER 2 outperforms ER 1.6 and other frontier models on Safety Instruction Following and Human Proximity tests, which evaluate adherence to physical constraints during reasoning and spatial awareness for detecting humans. In one demonstration, a humanoid robot halts autonomously when a person enters its workspace and resumes only when the area clears. Google DeepMind is also introducing a new benchmark to evaluate a foundation model's ability to act as a safe VLA orchestrator — testing enforcement of safety constraints, environmental monitoring, physical feasibility assessment, and human clarification seeking. Details are documented in an accompanying safety technical report that the company says will be released alongside the model.
Spatial reasoning has also broadened. Success/failure detection now operates on raw video feeds rather than static snapshots, catching mid-execution failures like spills, slips, or misalignments. General instrument reading extends beyond circular dials and sight glasses to include digital displays, linear scales, rulers, and liquid thermometers across 10 instrument types. Enhanced spatial visual question answering leverages Gemini's multimodal improvements. These capabilities matter for industrial inspection, laboratory automation, and any domain where robots must interpret diverse physical readouts in real time.
Our Take: A Meaningful Step Toward Embodied Agents, But Gaps Remain
Gemini Robotics ER 2 represents a credible advance in embodied reasoning. The emphasis on temporal intelligence — progress classification and moment-finding — tackles one of robotics' hardest problems: knowing when a task is actually done. The reported 91.3% moment-finding accuracy at sub-second latency is notable because it meets the practical threshold for closed-loop control. The multi-robot collaboration framework, while early, acknowledges the heterogeneous reality of robot fleets and provides a semantic handoff mechanism that could simplify multi-vendor deployments.
However, several caveats deserve attention. The 57.4% progress classification accuracy, while an improvement, indicates substantial room for growth before robots can reliably self-correct across the long tail of real-world variability. The safety benchmarks, though improved, are proprietary evaluations; independent replication on standardized testbeds will be necessary to gauge real-world robustness. Availability is also staged: the Gemini API and AI Studio offer immediate access, but the Enterprise Agent Platform remains in private preview, suggesting that production-grade orchestration tooling, monitoring, and compliance features are still maturing. Developers should treat ER 2 as a powerful research and prototyping platform today, with production deployment contingent on further validation in their specific environments.
FAQ
What is Gemini Robotics ER 2 and how does it differ from ER 1.6?
Gemini Robotics ER 2 is an embodied reasoning model that acts as a high-level brain for robots, enabling real-time video understanding, multi-step task planning, tool orchestration, and multi-robot collaboration. Compared to ER 1.6, it adds continuous progress classification (57.4% accuracy), precision moment-finding (91.3% accuracy, 0.96s mean absolute distance), 4x faster execution speed, and native multi-robot semantic handoff. It also integrates with the Gemini Live API for sub-second bidirectional streaming.
How can developers access and use Gemini Robotics ER 2 today?
Developers can access ER 2 via the Gemini API and Google AI Studio immediately. The Gemini Enterprise Agent Platform offers the model in private preview. Google provides example configurations and prompting guides on GitHub, including a demo that orchestrates Boston Dynamics Spot APIs for navigation and manipulation tasks.
What safety improvements does ER 2 introduce?
ER 2 achieves gains on Safety Instruction Following and Human Proximity benchmarks, demonstrating the ability to halt a humanoid robot when a person is nearby and resume autonomously once the area clears. Google DeepMind is also releasing a new benchmark for evaluating a foundation model's capacity to act as a safe VLA orchestrator, covering constraint enforcement, environmental monitoring, feasibility assessment, and human clarification seeking. A safety technical report accompanies the launch.
Which robots and platforms are compatible with ER 2's multi-robot collaboration?
The announcement demonstrates collaboration between Apptronik's Apollo 2 humanoid and Franka's F3 Duo arm. The framework is designed for heterogeneous fleets — wheeled rovers, humanoids, arms, and mobile manipulators — that can share semantic understanding to hand off subtasks. Compatibility depends on exposing low-level control interfaces (VLA models, navigation APIs) as tools that ER 2 can orchestrate.
What are the current limitations of Gemini Robotics ER 2?
Progress classification accuracy at 57.4% suggests robots may still misjudge task completion in complex or novel scenarios. Safety benchmarks are proprietary and not yet independently validated on standardized testbeds. The Enterprise Agent Platform remains in private preview, limiting access to production-grade orchestration tooling. Latency, while sub-second, may still be insufficient for the most dynamic high-speed manipulation tasks.
