Summary
Are legions of smart devices — fridges, ovens, vacuum cleaners, lawnmowers, security cameras, and TVs — conspiring to take over the internet? No. But they could soon be unwitting hosts to self-replicating AI models that spread from device to device and prove remarkably difficult to contain.
Anthropic’s self-replicating AI warning is alarming deja vu
Anthropic says its research found ‘signs of life for AI worms,’ that could colonize household devices. Experts say they have a point.
Anthropic says its research found ‘signs of life for AI worms,’ that could colonize household devices. Experts say they have a point.
Welcome to the unsettling world of AI worms, where, depending on who you ask, the prospect of models spreading uncontrollably across household appliances is either an imminent cybersecurity risk or another case of AI companies spreading doomer-tinged hype. In a blog post published this week, Anthropic warns that small AI models could be “months away” from acquiring such capabilities. Its experiments offer some evidence to support that, but leave a huge gap between what researchers have demonstrated and the looming threat they foretell. Still, experts who spoke to The Verge say that while computer worms and fears of self-replicating AI are nothing new, recent advances suggest there are good reasons to take the warnings seriously.
Self-replicating machines have long haunted AI safety researchers. They raise the prospect of systems spreading beyond human control that are difficult, if not impossible, to shut down. As with many fears about advancing AI, attention has largely focused on increasingly powerful frontier models, such as OpenAI’s Astra or Anthropic’s Mythos. But a key limiting factor is their size: They are colossal. Potentially containing trillions of parameters — values learned during training — frontier models require vast computing resources, largely confining them to data centers, where they can be more easily monitored and, in principle, switched off.
Smaller models, however, present a different problem. They’re rapidly improving, can run on consumer-grade hardware, and are often freely available to download. Anthropic’s researchers wanted to know whether one could run across a network of smaller, less powerful devices. They successfully split Google’s Gemma 4, a model with 26 billion parameters, across 15 Android smart TV boxes, none powerful enough to run it alone. It worked, albeit very slowly.
Having shown that a model could run across household devices, the researchers wanted to see whether AI could perform the necessary steps for self-replication in such an environment. They built a simulated network of computers designed to behave like Android devices, then unleashed an AI-powered “worm” — a type of malware that automatically replicates and spreads across networks — tasked with finding new machines, copying the AI model onto them, and getting it running on each. It’s worth noting that the model being propagated — Meta’s 8 billion parameter Llama 3.1-8B — was different to the one coordinating the attack, and much smaller.
Only the most sophisticated model tested, GLM-5.2 from China’s Z.ai, managed to spread across the entire network, though smaller models successfully pulled off individual steps. Based on how quickly they are improving, the Anthropic researchers predict that smaller models could equal GLM-5.2’s capabilities by the end of the year.
“As a prediction that goes beyond our experiments, we expect a future awash with agents running on all types of hardware, including some that aren’t under human control,” the researchers wrote. They acknowledged that they haven’t yet demonstrated “a full rogue replicator,” nor do they know what one might look like (they suggest a “slow, distributed ‘digital slime mold’” or “a swarm of fruit flies”). But they said their findings show “signs of life for AI worms,” and called for stronger security on the many devices a replicating system could use to spread.
“We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay,” they wrote. It is important to note that Anthropic did not use its own models for these tests and has repeatedly argued that open alternatives to its AI models carry “more risk” than closed systems.
The warning has sparked familiar accusations of PR hype and AI doomerism, including from readers of The Verge’s earlier coverage. After all, there is a considerable chasm between showing an AI can run slowly across a bunch of cheap TV boxes and a self-replicating AI worm tearing through household devices. But while interrogating a company’s motives is certainly worthwhile, allegations of deliberate fearmongering are hard to substantiate, particularly as Anthropic has long raised concerns about AI safety. Nor is Anthropic alone in investigating the threat of self-replicating AI, or calling for better security across connected devices.
Earlier this year, Palisade Research, an AI safety research firm based in California demonstrated that AI models could autonomously copy themselves onto other machines. “What’s new here is that these models were much smaller, in terms of parameter count, than previous ones which have been shown to be capable of self-replication,” Jeffrey Ladish, Palisade’s executive director, told The Verge. Anthropic’s work takes that another step further by showing that an AI can run across multiple low-powered devices. “This will also be true of big models,” Ladish said, predicting that even frontier models might eventually be able to run on large networks of consumer chips, such as those found in MacBooks.
“Prior research had already demonstrated the threat of AI-driven computer worms is realistic.”
Computer worms aren’t new either. They’ve been infecting machines for decades. But AI could make them smarter, more adaptable, and better equipped to tackle any obstacles. Ladish noted that even smaller models are getting better at carrying out cyberattacks autonomously. “Prior research had already demonstrated the threat of AI-driven computer worms is realistic,” University of Toronto professor Nicolas Papernot, who has studied them, told The Verge.
Taken together, the capabilities Anthropic demonstrated could dramatically expand the range of devices worth targeting in cyberattacks. “Devices that were not a target previously because of limited human attacker resources and time can now become targets, if not for the data they hold, for the purpose of using them as a launchpad to attack more valuable targets,” Papernot said. And there is no shortage of targets: the internet is riddled with poorly secured smart devices.
Papernot cautioned against anthropomorphizing AI, but said the threat of AI-driven computer worms is “concrete” and “deserves immediate action,” particularly when it comes to securing critical infrastructure like hospitals and power grids.
“Unless there is a massive effort to secure devices of all kind[s], it’s quite likely AI worms will have fertile grounds to colonize,” Ladish said. “I expect we’ll see the first of these in the wild in the next 12 months.”