Clustering Attacks on Web Apps to Find the Real Story Behind the Headlines

Clustering Attacks on Web Apps to Find the Real Story Behind the Headlines

Our goal of clustering attacks on web applications is two-fold

Security products which aim to block attacks may do their job perfectly while also reporting the attacks that are blocked. However, one of the biggest problems in the cyber security arena today is alert fatigue, where there are too many alerts to manually process. The largest data breach in history  affecting more than 41 million customer payment card accounts could have been prevented if the right action from the visible security alerts were taken. A web site protected by a web application firewall may be targeted by anywhere from hundreds of thousands of attacks to millions of attacks in a single day.

The amount of manpower given for processing and analyzing these alerts is always not enough, and the result is a flood of important data which is not handled and analyzed due to the alert fatigue. However, by leveraging artificial intelligence, we can develop sophisticated machine learning algorithms to cluster alerts to automate and consolidate those alerts, condensing days or weeks of work into minutes.

Our goal of clustering attacks on web applications is two-fold:

  1. Highlight interesting patterns inside the attacks
  2. Distill the massive amount of attacks to a few actionable incidents

Clustering can help us create a “story” out of the attacks (naming them based on behavior), making them more easily understood to a human observer and easier to analyze. For example, when seeing a cluster called, “SQL injection attack from several IPs in China using a Havij scanner,” the story behind it is much clearer than analyzing the thousands of attacks this cluster contains and trying to find the common pattern between them.

A simple “group-by” algorithm that takes the alerts and groups them by a specific attribute is not good enough. The reason is that there is not a single structure for attacks, and there is no single attribute which can define all the attacks. Thus, a more sophisticated algorithm, which considers a general distance function between attacks on web applications, is needed.

The algorithm has three main stages:

  1. Feature extraction
  2. Distance calculation
  3. Clustering of the attacks

Feature extraction

The raw data that enters the algorithm is an HTTP request that contains an attack stopped by the firewall, with some additional fields containing more data about the attack, like the source IP and the type of attack.

By leveraging our web application security domain knowledge, we extract additional meaningful features from the raw data that can help us describe the attack.

For example, the source of an attack is not defined solely by the IP. We also use geolocation services to extract more the about the origin of the IP, like source country, ISP, coordinates, ASN etc. It is also useful to know whether this IP comes from some kind of anonymity framework like TOR or an anonymous proxy.

Distance Calculations

The next task is to determine a way to calculate the distance between two attacks. This is a core stage of the algorithm as it determines when two attacks are similar, which in general is what the algorithm is trying to achieve. Calculating a distance between two points in the plane is easy – there is a precise formula to do it – but how can we calculate the distance between two URLs or two IPs?

We need to find a method to calculate the distance for every meaningful feature we have in our data, and then combine all these distances to find a single measure between two attacks.

Clustering of the Attacks

The final step is to take the data with all the extracted features and the distance measure between attacks to construct clusters of the attacks. In our case, we used a streaming clustering algorithm. This algorithm creates the clusters over time by receiving a stream of data as more and more attacks enter the system.

The importance of clustering in streaming mode is that the attacks are being delivered in real time. This method of stream clustering helps the performance of the algorithms in both time and memory, as not all the attack data is stored in memory all the time, only the current clusters with their unique features.

To conclude, clustering attacks on web applications help to understand the hidden patterns behind the attacks and to make huge amounts of data comprehendible to the human security expert. Constructing such a clustering algorithm requires more than just machine learning knowledge, it requires a high level of domain knowledge in cyber security to understand and construct the various parts of the algorithm.

To read more about clustering of attack on web applications see Imperva’s blog series.

Featured

  • New Report Reveals Top Trends Transforming Access Controller Technology

    Mercury Security, a provider in access control hardware and open platform solutions, has published its Trends in Access Controllers Report, based on a survey of over 450 security professionals across North America and Europe. The findings highlight the controller’s vital role in a physical access control system (PACS), where the device not only enforces access policies but also connects with readers to verify user credentials—ranging from ID badges to biometrics and mobile identities. With 72% of respondents identifying the controller as a critical or important factor in PACS design, the report underscores how the choice of controller platform has become a strategic decision for today’s security leaders. Read Now

  • Overwhelming Majority of CISOs Anticipate Surge in Cyber Attacks Over the Next Three Years

    An overwhelming 98% of chief information security officers (CISOs) expect a surge in cyber attacks over the next three years as organizations face an increasingly complex and artificial intelligence (AI)-driven digital threat landscape. This is according to new research conducted among 300 CISOs, chief information officers (CIOs), and senior IT professionals by CSC1, the leading provider of enterprise-class domain and domain name system (DNS) security. Read Now

  • ASIS International Introduces New ANSI-Approved Investigations Standard

    • Guard Services
  • Cloud Security Alliance Brings AI-Assisted Auditing to Cloud Computing

    The Cloud Security Alliance (CSA), the world’s leading organization dedicated to defining standards, certifications, and best practices to help ensure a secure cloud computing environment, today introduced an innovative addition to its suite of Security, Trust, Assurance and Risk (STAR) Registry assessments with the launch of Valid-AI-ted, an AI-powered, automated validation system. The new tool provides an automated quality check of assurance information of STAR Level 1 self-assessments using state-of-the-art LLM technology. Read Now

  • Report: Nearly 1 in 5 Healthcare Leaders Say Cyberattacks Have Impacted Patient Care

    Omega Systems, a provider of managed IT and security services, today released new research that reveals the growing impact of cybersecurity challenges on leading healthcare organizations and patient safety. According to the 2025 Healthcare IT Landscape Report, 19% of healthcare leaders say a cyberattack has already disrupted patient care, and more than half (52%) believe a fatal cyber-related incident is inevitable within the next five years. Read Now

New Products

  • A8V MIND

    A8V MIND

    Hexagon’s Geosystems presents a portable version of its Accur8vision detection system. A rugged all-in-one solution, the A8V MIND (Mobile Intrusion Detection) is designed to provide flexible protection of critical outdoor infrastructure and objects. Hexagon’s Accur8vision is a volumetric detection system that employs LiDAR technology to safeguard entire areas. Whenever it detects movement in a specified zone, it automatically differentiates a threat from a nonthreat, and immediately notifies security staff if necessary. Person detection is carried out within a radius of 80 meters from this device. Connected remotely via a portable computer device, it enables remote surveillance and does not depend on security staff patrolling the area.

  • Camden CV-7600 High Security Card Readers

    Camden CV-7600 High Security Card Readers

    Camden Door Controls has relaunched its CV-7600 card readers in response to growing market demand for a more secure alternative to standard proximity credentials that can be easily cloned. CV-7600 readers support MIFARE DESFire EV1 & EV2 encryption technology credentials, making them virtually clone-proof and highly secure.

  • Luma x20

    Luma x20

    Snap One has announced its popular Luma x20 family of surveillance products now offers even greater security and privacy for home and business owners across the globe by giving them full control over integrators’ system access to view live and recorded video. According to Snap One Product Manager Derek Webb, the new “customer handoff” feature provides enhanced user control after initial installation, allowing the owners to have total privacy while also making it easy to reinstate integrator access when maintenance or assistance is required. This new feature is now available to all Luma x20 users globally. “The Luma x20 family of surveillance solutions provides excellent image and audio capture, and with the new customer handoff feature, it now offers absolute privacy for camera feeds and recordings,” Webb said. “With notifications and integrator access controlled through the powerful OvrC remote system management platform, it’s easy for integrators to give their clients full control of their footage and then to get temporary access from the client for any troubleshooting needs.”