Report: 51% of web site traffic is 'non-human' and mostly malicious

Report: 51% of web site traffic is 'non-human' and mostly malicious

Summary: Web site analytics packages record what real people do on a site but most web site traffic comes from other computers often with nefarious intent.


Incapsula, a provider of cloud-based security for web sites, released a study today showing that 51% of web site traffic is automated software programs, and the majority is potentially damaging, -- automated exploits from hackers, spies, scrapers, and spammers.

The company says that typically, only 49% of a web site's visitors are actual humans and that the non-human traffic is mostly invisible because it is not shown by analytics software.

This means that web sites are carrying a large hidden cost burden in terms of bandwidth, increased risk of business disruption, and worse.

Here's a breakdown of an average web site's traffic:

- 5% is hacking tools searching for an unpatched or new vulnerability in a web site.

- 5% is scrapers.

- 2% is automated comment spammers.

- 19% is from "spies" collecting competitive intelligence.

- 20% is from search engines - which is non-human traffic but benign.

- 49% is from people browsing the Internet.

The data was collected from a sample of 1,000 websites that are enrolled in the Incapsula service.

I spoke with Marc Gaffan, co-founder of Incapsula. "Few people realize how much of their traffic is non-human, and that much of it is potentially harmful."

Incapsula offers a service aimed at securing small and medium sized businesses. It has a global network of nine data centers that analyze all traffic to a customer's site and blocking harmful exploits in real-time, while also speeding up page loading times through cached content closer to users.

"Because we have thousands of web sites as customers, we spot exploits way ahead of others and we can then block them for all our customers. That's the benefit of scale. We also maintain a virtual patch service that prevents harmful exploits days and sometimes weeks before a patch is ready."

There is no software or hardware installation required by the customer, a small change in a web site's DNS records directs traffic through Incapsula's data centers. And all analytics, and search engine rankings, are unaffected by the change.

Web sites are significantly faster because the company caches content and keeps it close to where users are located.

An important aspect of the service is that it is in compliance with the Payment Card Industry data security standard (PCI) which is essential for online merchants. They risk losing their ability to process credit card payments if they don't meet strict PCI requirements.

The company offers a free service for sites with less than 25 GB of monthly bandwidth, and premium plans start at $49 a month.

Foremski's Take: I'm curious to try this service because looking at my server logs I get hit by about 28 'robots' daily, and while some are from legitimate sources such as Google, Yahoo, Microsoft, the majority are unidentified and together, they use as much as one-third of my bandwidth.

This means that the human user experience suffers because my server is trying to deal with all the 'non-human' traffic generated by software programs hitting the site.

Incapsula's ability to block exploits before a patch is available is another attractive feature. I don't have time to keep up with the many security patches sent out, and then installing and upgrading multiple programs is a chore I'd rather do without.

More info: Incapsula In The News

Topics: Software Development, Browser

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • What OS / webserver vulnerabilities are they scanning for?

    You can tell by the URLs logged. I'll let "ye" fill in the details;-)
    Richard Flude
  • Your numbers on your headline and the article don't add up...

    49% is browsing. -benign
    20% is search engines -benign

    That's almost 70% of web traffic and is benign.
    That leaves only 30% malicious traffic, and while still bad is nothing like the implied 51% of all traffic is malicious.

    Where does e-mail, and other non-interactive background services fit in there?
    • Headline does not imply that.

      51% is non-human, and MOSTLY malicious.

      30% is more than half of 51%, and is the majority, if not "mostly"...
      • Huh?

        Do you know math?
        51% of the total is non-human.
        31% of the total is malicious.
  • WOW

    Whew! I am sure glad they cleared that up for us.

    In other news, President Obama distributes twinkies to toys for tots campaign, gas prices are extreme and the government could not be happier and Sachs Goldman tells you what they really are thinking behind those fake smiles.

    Slow news day ZD?
  • Increditable!

    This is sad! And it will not get any better, unless us humans demand it get better. It seem easier for the ISPs to just charge us more money rather than assess the real problem. They could easily do what this company is doing and give us the bandwith we are paying for. My soapbox for the week!
    • Hmmmm.....

      Shouldn't we get a proportionate 51% discount on our access fees? Hmmmmm???? It's kind of indefensible that we don't get SOME kind of break!
  • Non Human Web Traffic

    I wonder how much Twitter traffic is non human?
    • A whole new gambling industry ...

      .. based upon the aggregated time for a bunch of people to realise that they are not Tweeting with a real person!
  • China..

    I have worked in hosting company, and as far as i can tell most of malicious scans and exploits coming from china.
    • Or just routed through China?

      There's this thing called "transparent proxying", so it's actually impossible to know where traffic originates.
  • Like junk mail

    Remember when everyone got junk mail in their mailbox?
    Something like 60% or more of mail sent through the postal service was junk mail.
    It all just got more technological.
  • Between this and so-called "Privacy Policies"...we're dead meat!

    It is significant that there is so much machine chatter going on over the internet. Have you considered how much additional machine-driven chatter is occurring under the so-called "Privacy Policies" of every company that you do business with? Consider that privacy laws do not protect privacy, rather, they promote commercialization and monetizing our privacy through our mail, our phones, and, yes, our computers. I guess I'm just so damned fed-up with pseudo-protections that protect businesses and also motivate them to not look out for our best interests. Insidious.