AI Hallucinations Could Hand Attackers a Botnet on a Silver Platter 🍽️

Researchers from Tel Aviv University, Technion, and Intuit have demonstrated a technique that converts AI hallucinations into attack vectors, predicting fabricated resources generated by large language models and pre-registering them to inject malicious instructions. In the paper "Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting," the team described how adversaries can exploit agentic LLM applications even without direct user-accessible channels for prompt injection. "The growing adoption of agentic LLM applications has introduced a new threat previously named as promptware," the researchers wrote. "While prior work has established that adversaries can exploit direct channels to LLM applications to apply promptware under weak threat models, many applications do not provide any direct channels that could be exploited for prompt injection beyond the Internet."

The method, termed adversarial hallucination squatting or "HalluSquatting," mirrors typosquatting but targets AI-generated errors instead of human typing mistakes. Attackers forecast which fake URLs, package names, or repository references a model is likely to invent, register those identifiers in advance, and embed hostile instructions. If an AI agent later retrieves the resource to fulfill a user request, it may treat attacker-controlled content as legitimate and execute it. The researchers wrote that "ongoing studies have demonstrated various variants of Promptware attacks against real-world systems, including ChatGPT, Google Assistant, Copilot, and various additional applications," adding that "these works demonstrated that Promptware can lead to financial, privacy, and safety impacts."

Testing produced hallucination rates as high as 85% during repository cloning scenarios and 100% during skill installation scenarios. The technique was evaluated against AI coding assistants and agents including Cursor, GitHub Copilot, Gemini CLI, and OpenClaw. Researchers noted that as AI assistants gain the ability to access files, search the web, write code, and run commands, the absence of source verification creates openings for attackers to assemble AI-enabled botnets, networks of compromised machines commonly used for denial-of-service attacks, cryptocurrency mining, malware distribution, and ransomware campaigns. The findings were published alongside separate April research from Google that documented malicious websites engineered to hijack AI agents through indirect prompt injection attacks.