WormGPT Strikes Back: The Evolution of Uncensored AI Tools on the Cybercrime Underground

When OpenAI’s ChatGPT exploded into the public consciousness in late 2022, security experts warned of a double-edged sword. The same technology that could draft code and write poetry could also be manipulated to create phishing scams and malware—if it were uncensored. But for a time, safety guardrails held. Large language models (LLMs) refused to comply with prompts for illicit content. That barrier has now collapsed.

Enter the next chapter of WormGPT—a name that has become synonymous with uncensored, malicious AI tools used by cybercriminals. What began as a rogue experiment in 2023 has morphed into a decentralized, evolving brand of AI-powered cyberweapons, hosted and distributed via Telegram bots, underground forums, and custom APIs. According to new research from Cato CTRL, two new WormGPT variants—based on xAI’s Grok and Mistral AI’s Mixtral—are actively circulating on BreachForums, offering plug-and-play access to black hat capabilities.

“Responsible AI takes the sidelines in most cases, never moves beyond panel discussions,” warns Mayank Kumar, Founding Engineer at DeepTempo. "If these models are great in general understanding and will do good for humanity, they can be repurposed by bad actors to cause harm... that’s what is happening with WormGPT or jailbreaking highly capable models to create phishing tools, malicious code, and hacking tutorials."

WormGPT’s Origins: A Playbook for Abuse

Originally launched in June 2023 by a user named “Last” on Hack Forums, the first version of WormGPT was based on GPT-J, an open-source model developed by EleutherAI. It was explicitly designed to help cybercriminals automate phishing, malware creation, and fraud. It even had pricing tiers, ranging from €60 a month to a €5,000 private deployment.

By August 2023, WormGPT shut down—exposed by journalist Brian Krebs, who unmasked its developer as Rafael Morais. But its legacy didn’t end. Instead, it evolved. Like Hydra, cutting off one head spawned many more.

WormGPT Rebooted: Variants Powered by Grok and Mixtral

Cato CTRL’s threat research team infiltrated Telegram channels and underground forums to investigate new variants. Two users—keanu and xzin0vich—emerged as operators of WormGPT clones now used by thousands of cybercriminals.

• Keanu-WormGPT is powered by xAI’s Grok. It operates via a Telegram chatbot that reveals its true architecture through clever LLM-jailbreaking techniques. Researchers found the model employs a custom system prompt designed to override Grok’s built-in safety guardrails—letting it generate phishing emails, PowerShell credential-harvesting scripts, and more.

• xzin0vich-WormGPT uses Mixtral, a Mixture-of-Experts model developed by Mistral AI. The system prompt actively rejects default Mixtral behavior, instead forcing the bot into “WormGPT mode.” Technical clues, such as “top_k_routers: 2” and “kv_heads: 8,” further confirmed its Mixtral lineage.

The weaponization of these foundation models is stark evidence that cybercriminals are not waiting for open-source alternatives—they’re hijacking today’s commercial-grade AI through prompt injection, wrapper APIs, and fine-tuning with illicit datasets.

The Rise of the “Uncensored LLM-as-a-Service” Economy

What’s alarming is not just the existence of WormGPT variants—but their accessibility. Both keanu and xzin0vich offer subscription plans, integrate Telegram chatbots for instant interaction, and advertise to thousands of followers across underground platforms.

“WormGPT” has become less of a singular product and more of a brand—a category of jailbroken AI tools evolving to outpace content moderation and security controls. In its wake, alternatives like FraudGPT, DarkGPT, EvilGPT, and even XXXWolfGPT have proliferated. And the next wave is already here: fine-tuned, dark-web-trained LLMs built from scratch for cybercrime.

Enterprise Blind Spots: AI Threats No Longer Hypothetical

Kumar is blunt in his warning: "Two years ago, it was speculation—today it is sadly our reality. LLM-powered attacks are here and are avoiding traditional security defenses."

According to Cato CTRL, organizations continue to underinvest in defenses against AI-generated attacks. Meanwhile, AI now ranks as the most unaddressed threat in enterprise security—surpassing even ransomware. Three of the top six security risks flagged by CISOs involve LLM abuse, AI-generated deepfakes, and adversarial prompt engineering.

Security stacks relying solely on signature-based defenses, sandboxing, or perimeter firewalls are dangerously outdated. “If your cybersecurity stack relies purely on static defense signatures… your organization is exposed,” Kumar adds. "What's needed is intelligent anomaly-based, adaptive detection and deep visibility across systems, networks, and endpoints to catch fast-evolving LLM-powered attacks before they wreak havoc."

Fighting AI with AI: A New Defensive Imperative

Cato CTRL recommends deploying behavioral analytics, machine learning-powered threat detection, and fine-grained access control as critical defenses. Key strategies include:

• Anomaly-based Detection: Use XDR platforms with user and entity behavior analytics (UEBA) to identify malicious LLM activity that evades traditional filters.

• Zero Trust Enforcement: Apply dynamic, identity-aware routing and continuous device posture checks to contain potential breaches.

• Shadow AI Monitoring: Track unauthorized GenAI tool usage inside the organization through CASB dashboards.

• Phishing Simulations with GenAI: Train users using real-world phishing tactics generated by AI to simulate modern threat vectors.

Conclusion: The Pandora’s Box of Uncensored LLMs

The WormGPT saga is a case study in how quickly AI innovation can outpace regulation and security. It shows what happens when the open frontier of language models intersects with the black market of cybercrime.

The attackers have already adapted. Now, defenders must follow suit.

As Kumar puts it, "When we talk about great things these models can do, it’s equally important to discuss the other side." The darker side of LLMs isn’t theory—it’s operational, distributed, and monetized. And it’s already knocking on your firewall.

Disclosure: The information in this article is based on publicly available research and independent analysis. No LLMs were harmed—or jailbroken—in the making of this report.