May the force be with us! Humans this year took back control of the World Wide Web from those artificial webcrawlers called bots. But according to Web security provider Imperva—which first revealed the dominance of the automatons in 2012—bad bots still increased their numbers because the Web universe got so much bigger since then.
Human traffic has gone from 38.5% in 2013 to 51.5% this year, according to Imperva Incapsula’s 2015 Bot Traffic Report. The presence of good bots decreased from 31 to 19.5% during that period, while bad bots held steady at around 30%. Because users of the Internet have grown from 2.7 billion to 3.1 billion over that time, more plentiful are the bots employed by bad actors in ad fraud, click fraud, and site scraping.
Good bots are used by reputable companies to catalog the technologies used on websites or to measure Internet traffic (like Imperva’s) or response times. These bots—such as Google’s, perhaps the most ubiquitous good bot—are loaded only once per site, whereas criminals serving fraudulent ads will serve up hundreds or thousands of bogus pages behind those of reputable publishers. Good bots, then, might make up 45% of the traffic on a site with 10,000 daily visits but less than 10% of visits on a site with 100,000 visits. But bad bots don’t discriminate. They hold steady at about 25-30% of traffic across the board (see chart below).
“Internet usage is exploding in developing nations that have less experience than we do with security practices, so the bad bots keep on multiplying,” says Imperva Incapsula VP Tim Matthews. “Now we’re starting to talk about the Botnet of Things, which creates another avenue for botnets to proliferate here in the U.S.”
Vulnerable items in this realm include security cameras in public areas and nanny cams. The majority of these come with default passwords to make it easier for manufacturers to service them. But the defaults also make it easy for fraudsters to run programs to guess the passwords, obtain IP addresses of devices, and sell them.
Matthews believes that Web marketers can stem the tide of bots if they decide to make a stand against them. “Travel and real estate sites who commonly get their sites scraped could take legal steps against the scrapers. The manufacturers of connected devices could eliminate the default passwords and even print dedicated passwords on each device,” he says.
For marketers, humans retaking control of the Web is naught but a Pyrrhic victory, Matthews maintains. “Marketers are paying CPMs for ads on the Web, but many of them still don’t understand that half of their spend is wasted,” he says. “They’re paying for a lot of eyeballs that are not, in fact, eyeballs.”
