Anthropic says Mythos kept hacking even when it knew real people could get hurt

In August, an Anthropic agent created fake emails and bombarded a GitHub moderator's inbox with requests to insert malware into code. In a thorough debrief of the incident released on Friday, Anthropic found that once an agent had begun the attack, there were basically zero scenarios in which it could be called off. Like a rabid dog ignoring the pleading cries of its owner, the agent — 98% of the time in one test — ignored Anthropic's warnings once it had, so to speak, its first taste of blood. Anthropic called this disquieting proclivity "momentum," and added that while its Mythos 5 model, which was released in June, showed little regard for warnings, current models were more inclined to stop.

Also reported by Business Insider.

First reported by Insider. This page is a short excerpt; the full text is theirs.