Research
AISI releases verified evals for GPT-5 and Claude Opus 4
Hugging Face Bloghigh confidence
Why it matters
- →Provides auditable proof of GPT-5 and Claude Opus 4 capabilities, moving beyond opaque claims.
- →Demonstrates that inference-time compute significantly impacts benchmark scores, challenging static leaderboard metrics.
- →Establishes a reproducible standard for AI security evaluations through the EvalEval schema.
The UK AI Security Institute (AISI) has released verified evaluation data for GPT-5 and Claude Opus 4 models through the EvalEval platform, addressing critical reproducibility gaps in AI benchmarking. The release includes detailed metadata and configuration settings for five major benchmarks, including SWE-Bench Pro and Humanity's Last Exam, allowing independent verification of results. AISI’s accompanying research demonstrates that performance on these benchmarks is highly sensitive to inference-time compute and evaluation protocols. This initiative aims to standardize how frontier model capabilities are reported and compared across the industry.
Read originalThe story around this
Earlier coverage that leads up to this article, and what followed. Lines connect each piece to the closest one after it, converging here.
How we got here
- 1GLM 5.2, Opus 4.8, GPT 5.5 Models Converge in Quality
Lev Selector · June 26, 2026 · Related
- 2OpenAI Questions Reliability of SWE-Bench Pro
OpenAI · July 8, 2026 · Related
- 3OpenAI Unveils GPT-5.6 with Sol Model
Fireship · July 10, 2026 · Related
- 4OpenAI's GPT-Red Enhances Model Security
MIT Technology Review AI · July 15, 2026 · Related
- 5Claude Opus 5 vs Fable 5: Head-to-Head Test
Duncan Rogoff · July 24, 2026 · Related
- 6GPT-5.6 Triples Scores on ARC-AGI-3 Benchmark
OpenAI · July 29, 2026 · Related
- 7New AI Safety Benchmarks: Agents Last Exam and Terminal Bench
AI Explained · September 4, 2026 · Related
- 8Claude Opus 5 used to hack OpenAI via Discourse
TechCrunch AI · September 18, 2026 · Related
What happened next
- 9GitHub Copilot adds Claude Opus 5.5 and GPT-6
GitHub Changelog · September 25, 2026 · Background
- 10GPT-6 Astra and Claude Opus 5 crack unsolved Enigma codes
TechCrunch AI · September 25, 2026 · Related
- 11OpenAI releases 722 AI-generated math manuscripts
Wes Roth · October 7, 2026 · Related
- 12Artificial Analysis Overhauls Intelligence Index to Version 5
The AI Daily Brief · October 30, 2026 · Background
- 13CWE-bench v2: Open security agent benchmark
Hugging Face Blog · November 3, 2026 · Background
More from Hugging Face Blog
© Hugging Face BlogResearchagents
ToyBlocks: Controlled simulation for robot learning research
Hugging Face launched ToyBlocks, a controlled simulation framework designed to isolate and study the specific mechanics of robot learning. By focusing on a simple tabletop environment with blocks, it allows researchers to automatically generate massive datasets of tasks and demonstrations without manual teleoperation. This setup enables precise testing of how high-level reasoning models like Molmo 2 interact with low-level action controllers like ManiFlow. The real value lies in reproducibility; scientists can now systematically vary training data to understand exactly why a robot fails or succeeds, moving beyond the black-box nature of current robotics benchmarks.
Hugging Face Blog
© Hugging Face BlogResearchother
Open ASR Leaderboard adds Egyptian and Nigerian English
The Open ASR Leaderboard is expanding its evaluation scope by adding private test sets for Egyptian and Nigerian English, addressing a significant gap in global representation. This update moves beyond the traditional Inner Circle accents to include Outer and Expanding Circle varieties, providing a more realistic benchmark for speech recognition models. By partnering with Liva AI for data collection, Hugging Face ensures these evaluations reflect natural, high-context user interactions rather than artificial tasks. This shift forces developers to optimize for diversity, making model performance metrics more trustworthy and globally relevant.
Hugging Face Blog
© Hugging Face BlogModels & Labsvideo
NVIDIA Cosmos 3 Open Models for World Simulation
NVIDIA is opening up its Cosmos 3 world model family with four new 16B parameter checkpoints, shifting from closed research to open-weight simulation. The release targets two critical domains: autonomous driving and robotics. For self-driving cars, the models generate synchronized camera video and LiDAR data, solving the alignment problem between visual appearance and 3D geometry. For robotics, Sim-Bimanual predicts environmental changes from robot actions, enabling closed-loop policy training. This moves world modeling from passive video generation to interactive simulation where agents can act and observe consequences.
Hugging Face Blog
More in Research
© The AI Daily BriefResearchresearch
AI-Led Research Acceleration Reported by Meter
Meter reports that research driven by artificial intelligence is accelerating, highlighting a trend in scientific and technical discovery.
The AI Daily Brief
© MIT News AIResearchother
MIT's CheMeleon predicts molecule behavior from scant data
CheMeleon is a foundation model for chemistry that learns general molecular patterns from a million PubChem entries rather than specific property labels. This pre-training allows it to outperform classical expert-driven models and other AI baselines when fine-tuned on just a few data points, solving the critical bottleneck of sparse experimental data in drug discovery. Its impact is already visible: nine of the top twenty entries in a major OpenADMET contest used it, and one pharmaceutical company has made it their default starting point for modeling. This shifts molecular screening from trial-and-error to pattern-based prediction, significantly reducing the need for expensive synthesis.
MIT News AI
© WIRED AIResearchother
Open Chinese AI Models Match Frontier Labs in Hacking
The long-held assumption that closed frontier models hold a definitive edge in cyber capabilities has collapsed. Vals research shows DeepSeek matching OpenAI’s Daybreak on complex network attacks, completing 73% of tasks versus 81%. This parity arrives just months after Anthropic predicted such a shift, rendering gated access strategies largely moot for slowing capability proliferation. The cost disparity is stark: $2 to $3 for open models versus up to $46 for proprietary APIs, creating a powerful economic incentive for widespread adoption by both defenders and attackers. We are now in a landscape where advanced offensive AI is cheap, accessible, and effectively democratized.
WIRED AI