Scoop
AI tool news · rumor vs. reality

The AI Wire ●

Rumors tracked. Announcements verified. Updated daily.

ANNOUNCED

SemiAnalysis launches GDPx, a 206-task agent benchmark where the best model scores just 12% — Claude Opus 5.5 leads

SemiAnalysis released GDPx on November 2, a benchmark for general agentic capability spanning 11 domains the firm picked for where frontier models will earn their money: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. Version 1.0 has 206 tasks built with 14 specialist data partners, an eight-hour default time limit, and models running in their native harness — Claude Code for Claude, Codex for OpenAI's models, Antigravity for Gemini. New versions are planned every three to six months, with a public leaderboard at gdpx.semianalysis.com.

The headline number is how low the top score is: Claude Opus 5.5 leads at 12.0%, ahead of Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%. No model leads everywhere — Opus 5.5 wins quant trading, hardware and chip design, Carbon wins biology and edges software engineering, Astra wins recursive self-improvement and materials science — and under binary grading, finance and legal came back at essentially zero for every model. SemiAnalysis also notes Opus 5.5 and Astra are held back in cyber, biology and alignment by their own safety classifiers: strip those three domains out and Astra edges Opus 5.5, while Meta's Muse Wonder Full is rated the best available cyber model. Opus 5.5's lead is not cheap, at $51.41 per task against roughly $17 for Carbon.

Two caveats belong with the headline. GDPx is a paid SemiAnalysis product whose authors want it to become the benchmark of record, and its partner-built tasks are not independently audited. And a 12% top score reflects tasks deliberately chosen to be frontier-hard — it measures how far agents are from full automation of expert work, not whether they are useful today.

Sources

← Back to headlines