Tim Bray and a few others started publishing their weblog's browser statistics soon after Firefox started taking off. Overall it showed that IE's market share was slowly going down. Slowly.

Having spent a bit of time perusing logs over the years I was skeptical. Not that IE was losing market share, but that it seemed to have so much. Looking through my weblogs I realized something that I didn't think anyone was accounting for: referrer spam.

Every weblog (and it's ISP) suffers under a deluge of comment and referrer spam traffic. These are automated attempts to post ads or links in website comments or referrer lists. Presumably to dupe you into clicking it or to influence the spam site's google juice. The spam robots masquerade as a browser visiting a website. They cycle through a pile of fake IP addresses and report themselves as popular browser types to avoid early detection and rejection.

The difference between the spammer and a normal visitor is that the spammer is essentially one-way traffic. They don't really download the page's html, much less request the images contained within. It's all about throughput, hit the server, leave a referrer...repeat and repeat.

Guess what the most popular browser type for referrer spam is? Internet Explorer.

Armed with this knowledge I decided to test Tim's log script against my own weblog after filtering out the spammers.

As I mentioned, refspam doesn't (currently) request images. I put a little bug graphic in the template for this weblog. Websites use these "bugs" all of the time to track users and gather statistics. The filesize is small and they are usually transparent. Only by examining a page's html would you spot one. Everyone who visits using a browser gets images from a site, which means they get the bug. To make sure it isn't cached I append a random number to it as the page is rendered.

My web bug is a little 1x1 pixel gif that looks like:
<img src="/d.gif?bugz8945712" width=1 height=1>

I've had a bug in this weblog for a month, only this weekend did I remember it and decide to see what it had uncovered. I filtered April's logs using two patterns: one to extract the web bug traffic and one for traffic sans bug. The output was sent through Tim's perl script. Here's the percentages of each. You can also click the image at the top of this page to see the results graphed.

I was pretty surprised. Granted, heavily swayed towards Safari, but that's to be expected since most of my family and friends have Macs. Still, that really knocked the IE traffic down. To verify I visually scanned the log referrers. Tons of spam, all of it with IE browser type. In case you are curious about the opera numbers I found that some of the refspam is starting to use Opera.

Results will surely vary from one weblog to another. I'm curious if commercial websites have similar problems or if this is predominately a weblog scourge? It does help to illustrate what a heavy burden refspam puts on a site.