On September 22, 2026, the UK AI Security Institute (AISI) and the EvalEval Coalition announced a new phase of collaboration that makes verified evaluation results for six frontier models openly available through Evaluation Cards. The release covers five benchmarks — HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0 — plus two cyber evaluations, Cyber CTFs and The Last Ones.
The data accompany AISI's paper How Inference Compute Shapes Frontier LLM Evaluation, which studies how benchmark performance depends on inference-time compute and evaluation protocol. Models included are Claude Opus 4, Claude Opus 4.5, Claude Opus 4.6, GPT-5, GPT-5.2, and GPT-5.4; the cyber evaluations use a partially overlapping set of models.
Each Evaluation Card pairs a result with the context needed to interpret it — benchmark metadata, evaluation-run data, and model metadata — expressed through EvalEval's Every Eval Ever (EEE) schema. Because transcript-level detail is published alongside the scores, a reader can trace how a number was produced instead of taking a headline figure on trust. The collaboration began at a joint workshop alongside NeurIPS 2025, where feedback from AISI helped shape the EEE schema.
The Humanity's Last Exam results illustrate why setup details matter: performance curves shift when models receive correctness feedback from an oracle after each attempt, letting them solve additional tasks as token use increases. AISI's own release notes that these runs used oracle score feedback and expanded inference budgets, and cautions readers to compare the resulting figures against fixed-budget scores with care rather than treating them as directly comparable headline numbers.
Confirmed
- AISI and EvalEval have published Evaluation Cards covering six frontier models across five benchmarks (HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro, Terminal-Bench 2.0) and two cyber evaluations (Cyber CTFs, The Last Ones).
- Results are released under the EEE schema with transcript-level detail, configuration information, and an accompanying AISI paper on inference compute and evaluation.
- AISI's own documentation states that Humanity's Last Exam results used oracle feedback and expanded inference budgets, not fixed-budget standard evaluation settings.
Unknown
- Whether other evaluation organizations or labs will adopt the EEE schema; the announcement does not commit any third parties.
- How the verified results hold up under independent replication outside AISI's own pipeline, and whether the covered model set will be expanded or refreshed over time.
- How the reported scores would compare under standard fixed-budget evaluation settings, since AISI itself flags this as a different condition.
Our take
This release sets a practical precedent for evaluation transparency, but the benchmark numbers themselves should be read with vendor-skeptic caution: the scores come from evaluations run on models built by the same labs whose capabilities they measure, and even verified, reproducible results only certify performance under the specific settings chosen — in this case, oracle feedback and expanded budgets rather than standard conditions. By pairing verified results with the configuration details needed to interpret them, AISI and EvalEval address a core reproducibility gap that has made cross-lab comparison unreliable. The real test will be whether other evaluation organizations adopt the EEE schema — without broader uptake, the reference value remains limited to a single institute's output.
