Scam Sites at Scale: LLMs Fueling a GenAI Criminal Revolution

Cybercrime groups, like other businesses, can create more content in less time using GenAI tools. Over the last 6 months, Netcraft identified threat actors using these technologies across a range of attacks, from innovating advance fee-fraud to spamming out the crypto space. In total, our observations show LLM-generated text being used across a variety of the 100+ attack types we cover, with tens of thousands of sites showing these indicators.

Netcraft’s first party research into the use of generative artificial intelligence (GenAI) to create text for fraudulent websites in 2024 include:

  • A 3.95x increase in websites with AI-generated text observed between March and August 2024, with a 5.2x increase1 over a 30-day period starting July 6, and a 2.75x increase in July alone—a trend which we expect to continue over the coming months
  • A correlation between the July spike in activity and one specific threat actor
  • Thousands of malicious websites across the 100+ attack types we support
  • AI text is being used to generate text in phishing emails as well as copy on fake online shopping websites, unlicensed pharmacies, and investment platforms
  • How AI is improving search engine optimization (SEO) rankings for malicious content
  • July 2024 saw a surge in large language models (LLMs) being used to generate content for phishing websites and fake shops. Netcraft was routinely identifying thousands of websites each week using AI-generated content. However, in that month alone we saw a 2.75x increase (165 per day on the week centered January 1 vs 450 domains per day on the week centered July 31) with no influencing changes to detection. This spike can be attributed to one specific threat actor setting up fake shops, whose extensive use of LLMs to rewrite product descriptions contributed to a 30% uplift in the month’s activity.

    These numbers offer insight into the exponential volume and speed with which fraudulent online content could grow in the coming year; if more threat actors adopt the same GenAI-driven tactics, we can expect to see more of these spikes in activity and a greater upward trend overall.

    “As an AI language model, I can make scam emails more believable”

    Threat actors in the most traditional forms of cybercrime—like phishing and advance fee fraud emails—are enhancing their craft with GenAI.

    Netcraft observed signs of threat actors’ prompts being leaked in responses, providing insight into how they are now employing LLMs. In our Conversational Scam Intelligence service—which uses proprietary AI personas to interact with criminals in real-time—Netcraft has observed scammers using LLMs to rewrite emails in professional English to make them more convincing.

    Fake investment platforms are particularly well positioned for LLM enhancement, because the templates we’ve typically seen for these scams are often generic and poorly written, lacking credibility. With the help of GenAI, threat actors can now tailor their text more closely to the brand they are imitating and invent compelling claims at scale. By using an LLM to generate text that has a professional tone, cadence, and grammar, the website instantly becomes more professional, mimicking legitimate marketing content. That is, if they remember to remove any artifacts the LLM leaves behind.

    There’s no honor among thieves of course. Just as criminals are happy to siphon credentials from other phishing sites, Netcraft observed that when they see a convincing LLM-generated template, they may replicate the content almost verbatim. To evade detection and correct errors in the original template, some threat actors appear to be using LLMs to rewrite existing LLM-drafted text.

    Threat actors are becoming more effective at using GenAI tools in a highly automated fashion. This enables them to deploy attacks at scale in domains where they don’t speak the target language and thus overlook LLM-produced errors in the content. By example, Netcraft came across numerous websites where page content itself warns against the very fraud it’s enabling.

    It’s no surprise that threat actors are beginning to utilize GenAI to both create efficiencies and improve the effectiveness of their malicious activities. Netcraft has been observing this trend for some time and developing suitable countermeasures in response. Netcraft’s platform flags attacks with indicators of LLM-generated content quickly and accurately, ensuring customers get visibility of the tactics being used against them.

    For the complete research report visit here.

    Featured

    • Report: Cyber Attackers Continue to Turn to AI-Based Tools to Avoid Detection

      Comcast Business recently released its 2025 Cybersecurity Threat Report, a comprehensive analysis of 34.6 billion cybersecurity events detected between June 1,2024 and May 31, 2025. Now in its third year, the report offers business leaders a unique perspective into the evolving threat landscape and provides actionable insights to help organizations strengthen their defenses and align cybersecurity with business risk. Read Now

    • Axis Communications Creates AI-powered Video Surveillance Orchestra

      What if cameras could not only see the world, but interpret it—and respond like orchestra musicians reading sheet music: instantly, precisely, and in perfect harmony? That’s what global network technology leader Axis Communications set to find out. Read Now

    • Just as Expected

      GSX produced a wonderful tradeshow earlier this week. Monday was surprisingly strong in the morning, and the afternoon wasn’t bad at all. That’s Monday’s results and asking attendees to travel on Sunday. Just a quick hint, no one wants to give up their weekend to travel and set up an exhibit booth. I’m just saying. Read Now

      • Industry Events
      • GSX
    • NOLA: The Crescent City

      Twenty years later we finds ourselves in New Orleans. Twenty years ago the aftermath of Hurricane Katrina forced exhibitors and attendees to look elsewhere for tradeshow floor space. Read Now

      • Industry Events
      • GSX
    • Nothing Artificial About this Intelligence

      I have been looking forward to this year’s GSX show in New Orleans, the Cresent City, or if you prefer The Big Easy. It seems like quite a while since we’ve been here. Twenty years ago, ASIS, as it was known then was literally washed out of the city by someone known as Katrina. It is a good thing to come back to NOLA. Read Now

    New Products

    • Luma x20

      Luma x20

      Snap One has announced its popular Luma x20 family of surveillance products now offers even greater security and privacy for home and business owners across the globe by giving them full control over integrators’ system access to view live and recorded video. According to Snap One Product Manager Derek Webb, the new “customer handoff” feature provides enhanced user control after initial installation, allowing the owners to have total privacy while also making it easy to reinstate integrator access when maintenance or assistance is required. This new feature is now available to all Luma x20 users globally. “The Luma x20 family of surveillance solutions provides excellent image and audio capture, and with the new customer handoff feature, it now offers absolute privacy for camera feeds and recordings,” Webb said. “With notifications and integrator access controlled through the powerful OvrC remote system management platform, it’s easy for integrators to give their clients full control of their footage and then to get temporary access from the client for any troubleshooting needs.”

    • AC Nio

      AC Nio

      Aiphone, a leading international manufacturer of intercom, access control, and emergency communication products, has introduced the AC Nio, its access control management software, an important addition to its new line of access control solutions.

    • Camden CM-221 Series Switches

      Camden CM-221 Series Switches

      Camden Door Controls is pleased to announce that, in response to soaring customer demand, it has expanded its range of ValueWave™ no-touch switches to include a narrow (slimline) version with manual override. This override button is designed to provide additional assurance that the request to exit switch will open a door, even if the no-touch sensor fails to operate. This new slimline switch also features a heavy gauge stainless steel faceplate, a red/green illuminated light ring, and is IP65 rated, making it ideal for indoor or outdoor use as part of an automatic door or access control system. ValueWave™ no-touch switches are designed for easy installation and trouble-free service in high traffic applications. In addition to this narrow version, the CM-221 & CM-222 Series switches are available in a range of other models with single and double gang heavy-gauge stainless steel faceplates and include illuminated light rings.