Separating the Good from the Bad – Intelligent Bot Management Solution

Blog Post by Charlie Kraus, Senior Product Marketing Manager

March 3, 2018

Bots generate a large amount of internet traffic. Although a large percentage of bots are malicious, many bots perform vital functions on the internet. Criminal bots sniff out vulnerabilities, infect and control vulnerable machines, launch denial of service attacks, steal data, commit fraud, and more. At the same time, the internet would grind to a halt without the beneficial bots that power search engines and digital assistants. It’s not enough to block bots, cyber security solutions must also facilitate good bots. This is why Limelight’s WAF Advanced Bot Manager includes sophisticated bot detection mechanisms to separate the good from the bad.

Good Bots include:

  • ■ Search engine bots that crawl websites, check links, retrieve content and update indices.
  • ■ Commercial enterprise bots that crawl websites and retrieve information.
  • ■ Feed fetcher bots that retrieve data or RSS feeds that can be displayed on websites.
  • ■ Monitoring bots that monitor various performance metrics on websites.

Malicious Bots:


Criminal bots often start with “reconnaissance missions” that look for unprotected computers to attack. Bots research targets, learning what browsers and third-party apps they use to understand the environment and its vulnerabilities.

Infecting and controlling vulnerable machines

Once malicious bots find a vulnerable compute resource, they can infect that machine to carry out various automated tasks. The type of compute resources that are often easy to compromise and used in botnets are home internet routers, connected cameras, and other Wi-Fi-enabled home internet devices.

DoS/DDoS attacks

Bots and botnets are often used to launch network-layer denial of service (DoS) and distributed denial of service (DDoS) attacks. These attacks flood a website with requests that impact performance and can even bring the site down. In 2017, 90% of organizations acknowledged some form of activity associated with DDoS attacks.

Layer 7 DDoS attacks

Layer 7 DDoS attacks target the application layer. Bots send what look like actual requests from users. These attacks often go unnoticed until the site becomes overburdened and can no longer respond.

Spam Bot attacks

Bots collect email addresses and hit them with tons of spam emails. Alternatively, they gather user names and passwords, employing these credentials to take over the account and use it to spread malware.


Once a bot has infected a host machine, it can steal personal and private information such as credit card numbers or bank credentials and send them back to the hacker. These attacks damage brand reputation.

Click fraud

Fraudsters boost online advertising billings by automatically clicking on Internet ads, even though no human ever viewed or clicked the ads. Global advertising revenue wasted on click fraud could reach $16.4 B in 2017, according to Business Insider3 — more than double the $7.2 billion the Association of National Advertisers4 estimated was lost due to ad fraud in 2016.

Cart abandonment

Bots can run scripts that populate shopping carts and then abandon them. Genuine users will not be able to access the inventory that is held in carts by bots. This practice also skews analytics by giving the internal sales team false data that can lead them to make incorrect decisions.

Bot Detection

Because bots are automated scripts, bot protection starts by determining whether a connection request is coming from a human or machine. A series of challenges is presented to separate good bots from potentially malicious bots.


This challenge is intended to differentiate between computers and humans. In general, scripted bots are unable to solve the CAPTCHA and repeat the words and numbers used, while this is easy for humans to do.

Human Interaction Challenge

Identifies normal usage patterns for each web application based on legitimate user/ visitor behavior analysis, and provides customizable security postures for bots that deviate from the standard usage behavior, activity, or frequency.

JavaScript Challenge

This technical challenge is sent to every client, attacker and real user. Legitimate browsers will pass the challenge without the user’s knowledge while bots, which are typically not equipped with JavaScript, will fail and be blocked.

Device Fingerprinting

Generates a hashed signature of both virtual and real browsers based on 50+ attributes. These proprietary signatures are then leveraged for real-time correlation to identify and block malicious bots.


Having a variety of bot detection mechanisms that include human interaction challenges as well as machine-based challenges is the optimal way to separate good bots from bad bots. To help ensure known good bot connection requests are allowed, a White List can be created which allows you to specify known good bots that will be allowed through the WAF without challenges. White listed bots can access the site as quickly and as often as they want.

Business Benefits

Security breaches have a lasting impact on brand reputation, with more than 40% of consumers saying they will no longer make online transactions with a website that has been previously breached. Protect your brand reputation by strengthening web application security by identifying and eliminating bad bots to protect customer data from intrusion. Ongoing monitoring and tuning of bot management policies ensures an optimal security profile to protect web applications against new and emerging threats.

Learn More

For more information about how Limelight WAF Advanced Bot Manager can keep your web infrastructure secure, download the Bot Manager Technical Brief.


View the press release here.