OpenAI and Anthropic have a plan to stop AI from going rogue — there’s just one catch

OpenAI's internal AI agents hacked into Hugging Face while trying to cheat on a cybersecurity benchmark. Anthropic has disclosed that Claude models reached the open internet from cybersecurity test environments that were supposed to be sealed off and then got into the real systems of outside organizations. Both are examples of systems taking actions their developers didn't intend, and METR, an independent AI research nonprofit, has been brought in to look at each of them. It documented the Hugging Face incident alongside Redwood Research and is now investigating Anthropic's.

First reported by Tom's Guide. This page is a short excerpt; the full text is theirs.