OpenAI dissolved the team built to catch catastrophic AI risks, reassigning its work to other groups

OpenAI shut down its "Preparedness" team at the end of July. The team evaluated whether the company's AI models could pose serious or catastrophic risks, the Financial Times reports, citing internal sources. Its work on biological and cyber risks has been parceled out to existing teams.

Former unit lead Dylan Scandinaro now focuses on safety risks from "recursively self-improving" AI, systems that can optimize themselves and train other models. Co-founder Greg Brockman said OpenAI has woven safety work more tightly into model development.

Several safety staffers have left recently, including Chief Ethics Officer Chloe Bakalar and Joshua Achiam. Internally, unease is building. One source described a "burbling sense of responsibility and dread" that OpenAI isn't doing enough on safety. Employees have also spoken up publicly, especially after the autonomous hacking incident involving Hugging Face. One employee said he hoped OpenAI would treat it as a "warning shot."

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.