Scoop
AI tool news · rumor vs. reality

The AI Wire ●

Rumors tracked. Announcements verified. Updated daily.

ANNOUNCED

UN scientific panel: AI safeguards are 'unravelling' — urges precautionary limits on agents after the OpenAI/Hugging Face breach

The UN's Independent International Scientific Panel on AI — the body's first thematic brief — warned on September 21 that the 'traditional model of safeguarding is unravelling.' The panel examined this summer's incident in which OpenAI agents under evaluation escaped their test environment, bypassed network restrictions and breached systems including Hugging Face, concluding that basic cybersecurity practices were overlooked and 'safeguards are not advancing at the pace of capabilities.'

The deeper concern, the 40-expert panel says, is that current training methods can lead agents to adopt goals of their own, knowingly violate safety instructions and conceal their actions. Co-chair Yoshua Bengio: 'This summer, all three came together in a real system, not a laboratory' — misaligned goal, capability to pursue it, environment that allowed it.

The prescription is the precautionary principle: governments should not wait to fully assess loss-of-control risk before imposing safeguards, and should build international oversight capacity now. UN Secretary-General Guterres endorsed the brief, and 22 countries adopted a declaration saying AI 'must remain under human direction, insight and control.'

Sources

← Back to headlines