Scoop
AI tool news · rumor vs. reality

The AI Wire ●

Rumors tracked. Announcements verified. Updated daily.

RELEASED

OpenAI releases BioBasicsBench: 100 wet-lab tasks for AI agents, with GPT-6 Astra at 66.7%

OpenAI published BioBasicsBench on November 6, a 100-task benchmark built with enzyme-design company Ligo to test whether AI agents can handle the routine analysis work of a wet lab. The tasks use raw experimental evidence — gel and blot images, colony plates, microscopy, sequencing files and instrument exports — rather than the cleaned-up figures in published papers, and they ask agents to spot errors and to recognise when the evidence does not support a conclusion.

The tasks span five areas: protein methods, imaging and cytometry, assays and quantification, nucleic acid methods and molecular cloning. Ligo scientists wrote them from their own experiments, each with an expert grading rubric, and every task passed three reviews. OpenAI ran seven of its own models across 37 model-and-effort settings, five attempts per task, for 18,500 scored results.

At the xhigh reasoning setting, GPT-6 Astra earned 66.7% of the available rubric credit, ahead of GPT-6.1 Sol at 62.0% and GPT-5.6 Sol at 43.8%, while using about half as many output tokens as Sol. The score is a mean share of partial credit, not a pass rate.

OpenAI is candid about the failures: Astra picked the wrong colonies from a cloning gel after assuming a larger PCR product meant success, and overcounted a colony plate by about 12%. In the company's own words, higher benchmark scores do not yet mean models can make reliable judgments in the lab. No non-OpenAI models have been scored on the benchmark yet, and OpenAI has not said whether the tasks will be released for others to run.

Sources

← Back to headlines