How Bots and Bad Actors Bypass Web Application Firewalls (WAFs)
Jeffrey Edwards
|Cyber Risks & Threats | November 22, 2023
A Web Application Firewall (WAF) is a crucial security tool, designed to filter out malicious traffic through a set of predefined rules. Security professionals have traditionally relied on WAFs to defend against familiar threats, such as known malicious user agents and blacklisted IP addresses. However, the effectiveness of WAFs in the current cybersecurity landscape is increasingly questioned.
But as bots become increasingly sophisticated, a question arises: Are WAFs still equipped to discern the nuances between human users and automated bots? The answer, more often than not, leans towards no. The issue is inherent to WAFs’ design, which focuses on known threats but falls short in real-time detection and response to the complex, adaptive bots that now dominate the cyber threat horizon.
How WAFs Work: The Basics
Web Application Firewalls (WAFs) are designed with a specific goal: to shield web applications from attacks that exploit common software vulnerabilities. Their primary task is to analyze web traffic via both GET and POST-based HTTP requests and then enforce a set of predefined rules or policies to identify and filter out suspicious traffic that matches known attack signatures. In essence, they act as a filter, separating safe traffic from potential threats. To accomplish this, WAFs typically rely on one of three security models: negative, positive, or hybrid.
- Negative Security Model: Also known as a “deny list,” this model blocks requests that match known attack signatures. It operates on the principle of allowing all traffic except for what is explicitly identified as malicious based on predefined signatures of known attacks. This model is effective against familiar threats but might not catch new or unknown attacks.
- Positive Security Model: Known as an “allow list,” this model permits only requests that match a known pattern of good behavior. Everything that does not fit this pattern is blocked. This approach is generally more secure than the negative model because it can block even zero-day attacks. However, it can be more restrictive and may inadvertently block legitimate traffic if not carefully configured.
- Hybrid Security Model: This model combines elements of both negative and positive models. It first uses signatures to filter traffic (negative model) and then checks if the remaining requests match the pattern of known good behavior (positive model). This approach aims to balance security with usability, trying to block malicious traffic while minimizing false positives.
WAFs can be set up in various ways. They can be cloud-based, host-based, or network-based and are often deployed through a reverse proxy, meaning they stand between the user and the server, inspecting traffic before it reaches the web application.
Where WAFs Come Up Short
While WAFs are effective against known threats, their reliance on static, predefined rules can be a limitation. WAFs are designed to protect websites or web apps from known attacks, such as SQL injections, session hijacking, and cross-site scripting. They use a set of rules that filter out good bot traffic from bad bot traffic. In particular, WAFs look for requests that carry familiar attack signatures.
As a result, WAFs can only block familiar threats. They’re ineffective for blocking today’s ever-changing, advanced bots that don’t carry obvious attack signatures. In addition, many bot attacks, such as account takeover fraud, remain within perfectly normal business logic. It just looks like someone is trying to log in, which a WAF won’t recognize as a potential problem.
WAFs also rely heavily on IP reputation to manage bots. If the IP reputation of a request is bad, it assumes all activity from that IP will be bad. Conversely, if the IP reputation is good, it is likely to let all requests coming from that IP through. As mentioned in previous articles, bot operators can now rotate high-quality, residential IPs cheaply and easily, making a WAF an ineffective solution to detect and prevent bots.
Three Ways Sophisticated Bots and Bad Actors Bypass WAFs
Evasion Tactics: IP Spoofing and Rotating IPs
The simplest way to evade a static ruleset is to simply avoid it. To do so, hackers often engage in IP spoofing to disguise their network identity by altering the IP address in the packet headers. This allows them to masquerade as a trusted source, bypassing IP-based filtering rules of a WAF. Attackers may also use botnets or proxy services that rotate through a wide range of IP addresses. Each request appears to come from a different source, rendering IP-based blocking ineffective. WAFs that rely heavily on IP reputation systems are particularly vulnerable to this tactic. As IPs keep changing, it becomes challenging for the WAF to detect and block malicious traffic consistently. This method is especially prevalent in distributed denial-of-service (DDoS) attacks and automated web scraping, where numerous requests are sent from varied IPs, overwhelming the WAF’s ability to track and block all malicious sources.
Concealing Malicious Traffic with SSL Encryption
Attackers exploit SSL encryption to bypass Web Application Firewalls (WAFs) by hiding malicious payloads within encrypted traffic. Since SSL/TLS encryption secures the content of data packets during transmission, WAFs without SSL decryption capabilities cannot inspect the encrypted content. Malicious actors leverage this by encrypting harmful requests, knowing that such traffic will pass through the WAF without being analyzed for threats.
Exploiting Scanning Constraints by Padding Request Size
Attackers circumvent Web Application Firewalls by exploiting limitations in their scanning capabilities, particularly regarding request size. Knowing that many WAFs only scan a finite amount of bytes within a request, attackers deliberately create oversized requests. Typically, real-world HTTP(S) GET or POST requests are only a few hundred bytes to maybe 1-2 kilobytes in size. However, attackers pad these requests – for example, with large headers, cookies, or POST body text – to exceed sizes that WAFs efficiently scan, often going beyond 8 kilobytes. This tactic effectively bypasses WAFs, as many are not configured to scan or log anomalously large requests. This oversight is partly due to the computational expense and increased costs associated with scanning larger requests. Consequently, some WAFs, in an effort to be cost-competitive, do not enable enhanced scanning features by default, leaving web applications and APIs vulnerable to such padded attacks.
Towards a Future-Proof Web Security with CHEQ’s On-Site Security
The limitations of traditional Web Application Firewalls, particularly against sophisticated bots and advanced cyber threats, underscore the need for a more dynamic solution. CHEQ’s On-Site Security (OSS) offers a ‘GTM Native’ protection layer specifically designed to combat these challenges. Unlike conventional WAFs, OSS excels in real-time detection of malicious activities, ensuring robust defense against a wide array of threats, including those that exploit the inherent weaknesses of traditional WAFs.
OSS stands out by offering advanced security features while minimizing friction to customer journeys. It integrates seamlessly with your GTM tech stack, enhancing your security posture without compromising user experience. With OSS, your site transforms into a ‘safe zone,’ protecting both your key site assets and your legitimate visitors from threats like malicious scrapers, scalper bots, and sophisticated botnets.
Discover Advanced Protection with CHEQ’s On-Site Security
Elevate your cybersecurity strategy with CHEQ’s On-Site Security. Reach out to us for a detailed demonstration and see how our solution can seamlessly blend with your existing infrastructure, offering unparalleled protection against evolving digital threats.
Author
Jeffrey Edwards
Content Marketing Manager
Jeff is the resident content marketing expert at CHEQ. He has several years of experience as a trained journalist, and more recently in his career found a knack for communicating complex cybersecurity topics in an approachable yet detailed manner.
