On August 27, 2026, Google DeepMind announced what it describes as the world's first double-blind evaluation of a proprietary, frontier-class AI model. The pilot ran a Gemini 2.5 Flash-Lite model against confidential benchmarks inside a cryptographically secured environment, where neither side could see the other's protected asset.
The company partnered with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons on the test. The goal: prove that an outside evaluator can rigorously test a frontier model without the model provider ever seeing the test prompts — and without the evaluator ever touching the model weights.
The contamination problem
Google DeepMind frames the issue with an exam analogy. A student who has seen the questions in advance can post a perfect score that says nothing about what they actually know. The same applies to AI models: if a model has encountered benchmark prompts during training or fine-tuning — a problem known as benchmark contamination — its scores can be artificially inflated and stop reflecting real capability.
Until now, high-stakes external evaluations forced a tradeoff. Either the evaluator handed over test prompts, risking that the provider saw the questions in advance, or the provider handed over model weights, risking its intellectual property. Contractual safeguards and zero-logging protocols have long kept external prompts confidential, but the pilot adds a technical and cryptographic layer on top of those procedural guarantees.
How the double-blind setup works
The evaluation ran inside a secure enclave — a hardware-isolated trusted execution environment (TEE) — using Confidential Space from Google Cloud's Confidential Computing portfolio. The hardware attests to exactly what code will run before either party's assets enter the enclave, and both sides review and approve that code. The enclave then executes it and releases only the agreed outputs to the agreed recipients.
- Google DeepMind supplied the proprietary Gemini 2.5 Flash-Lite model; its weights were never exposed to outside parties.
- MLCommons supplied never-before-used prompts from its AILuminate safety benchmark family.
- AVERI encrypted the prompts, jointly ran the evaluation with Google DeepMind using software originally developed by OpenMined, then decrypted and graded the outputs against AILuminate criteria.
- Google DeepMind could not store or train against the MLCommons prompts; AVERI, OpenMined, and MLCommons could not see the model weights.
AVERI conducted its part of the work in July and August 2026 and produced a qualitative and small-scale quantitative assessment of the model, delivered to Google DeepMind as a confidential report. The team chose AILuminate because it represents a broad understanding of model reliability, though any benchmark could run in the same setup subject to pre-agreed length and turn restrictions.
From proxy to production model
The mechanism itself is not entirely new. In a 2024 pilot, OpenMined collaborated with Anthropic and the UK AI Security Institute to run a joint evaluation inside an NVIDIA H100 GPU enclave — but that test used stand-ins: GPT-2 as a proxy for a frontier model and a five-row evaluation as a proxy for a confidential biosecurity benchmark. The secure computation took just over one minute.
The new pilot moves from proxies to a production model. According to AVERI, it is the first end-to-end demonstration that a proprietary model from a leading AI company can be evaluated against a real benchmark with specific privacy guarantees for both parties.
Why it matters
The timing is regulatory as much as technical. Under the EU AI Act's General-Purpose AI Code of Practice, signatories must ensure qualified independent evaluators can assess models with adequate access, information, and time. In the United States, Illinois' SB 315 will require annual third-party audits of large frontier developers starting in 2028, and other state and federal proposals could mandate audits sooner. These laws demand that security and privacy be protected during audits but say little about how — a gap secure enclaves are positioned to fill.
Google DeepMind says the approach matters most for highly sensitive evaluations, such as cybersecurity testing or assessments run by government bodies, where handing prompts to a model provider is not an option. The company frames the pilot as a step toward a new frontier of model oversight, though it remains exactly that: a pilot, with a small-scale quantitative component and results shared only as a confidential report.
Our take
The real significance here is structural, not benchmark-related. If enclave-based evaluation holds up, the leverage shifts toward auditors: a frontier lab can no longer argue that independent testing requires surrendering either its weights or the evaluator's prompts. What remains unproven is scale — one Flash-Lite model against one benchmark family is a long way from continuous, multi-benchmark auditing of flagship systems, and the confidential nature of the findings means outsiders must take the process on trust rather than the results.
Series: 1. Koray Kavukcuoglu Named DeepMind SVP as Hassabis Becomes Chair · 2. Google DeepMind pilots a double-blind AI model evaluation · DeepMind Leadership
