9

Last month I noticed that the direct visits on our site have increased by nearly 150% whilst bounce rate is also considerably up.

After drilling down further I can see that we have had nearly 2000 direct visits from one town in Connecticut called Stamford, with a bounce rate of 100%!

I have been scratching around for answers but I can only find that it may be to do with our uptime monitoring tool; Pingdom.

Does anyone know/have any experience with this kind of issue, any help is appreciated

  • I have just noticed that we are receiving identical traffic in a town in England and a town in Scotland... This definitely makes me think it's to do with our uptime monitoring tool.
6
  • Sombody may have bookmarked any of your articles in social networking sites giving short term traffic
    – AgA
    Commented Dec 3, 2012 at 12:11
  • I doubt it, we've had almost 2000 visits from Stamford, Edinburgh and Southend on Sea, all of which have 100% bounce rates. Also the visits are to our homepage as opposed to any blog articles/answers that may warrant a bounce. Commented Dec 3, 2012 at 12:34
  • Might you be willing to disclose the name of the website? I have an idea, but need to know that piece of information. Just the domain name, not even the ccTLD, if you'd rather not say.
    – Ellie K
    Commented Dec 4, 2012 at 19:33
  • Hi, here is a bitly link to my website (don't want to paste the domain for search reasons!) bit.ly/WIHw2G Commented Dec 5, 2012 at 9:42
  • Jack I posted an answer with a link to a site discussing filtering the bots out.
    – Anagio
    Commented Dec 6, 2012 at 21:09

5 Answers 5

11

Determining the cause of strange localised traffic spikes that don't appear to be from human visitors requires patience and detective work, but the basic steps are to:

  1. Find out where it's coming from using the tools available to you (see below).

  2. Determine if it's dangerous or not by analysing the sources of the request, the frequency, the request headers themselves, and the effect on your network and server(s).

  3. Decide whether to block it in Google Analytics alone just to prevent it from appearing in your stats (using a Google Analytics filter), to block it at the server to prevent it ever reaching your site in future (using a firewall or a service such as Cloudflare), or to ignore it altogether.

I won't detail the security implications here because you seem more concerned about the effect it's having on your analytics data. Here are some observations about this particular case, with some pointers to determine the possible source and filter out unwanted traffic in Google Analytics:

It's probably not Pingdom

The Google Analytics spikes are unlikely to be visits from Pingdom for two reasons:

  • Pingdom's HTTP checks don't parse JavaScript, so they won't trigger embedded Google Analytics scripts. (I double-checked this with Pingdom's support team yesterday, because I use Pingdom too and was curious. Here's a direct quote from their support email: "As our HTTP check doesn't execute any JavaScript code, Google Analytics won't have any records of our visits done by our probe servers to your website.")

  • Pingdom don't currently have servers in Edinburgh. They list all of their Probe servers with the IP addresse and location on a page in your control panel (the link named "Probe servers" in the bottom right of all Pingdom admin pages when you're logged in). The only two listed in the UK right now are in Manchester and London.

But you don't have to take my word for it. If the spikes are ongoing and you want to confirm that Pingdom is not the cause of the sudden visits and bounce rate spikes, you could:

  1. Pause the Pingdom requests to your site and see if the hits from those areas disappear. (I suspect it won't make a difference.)

    — or —

  2. Set up a page at example.com/pingdom/any-url-you-like, point your Pingdom requests to that page instead of your homepage, and exclude the /pingdom/ subdirectory from your Google Analytics results by (a) removing the Analytics tracking code on that page, or (b) using a filter to exclude the page in Google Analytics if you can't remove the tracking code on a per-page basis (e.g. if you're using a CMS that includes the code on every page, and you don't have enough access to alter this behaviour).

How to filter requests with Google Analytics

This is only necessary if you suspect that Pingdom does parse JavaScript, and there's no indication that it does, but filtering out visits from Pingdom like this is one way to check if you want to reassure yourself. It's also useful to learn how to do this so you can filter out other types of visits in future if you need to:

  1. Log in to Google Analytics and click the "Admin" link (top right, next to "Help")

  2. Click the domain name or site name for the profile you want to filter. (e.g. example.com)

  3. Click the "filters" tab.

  4. Click "+ New Filter".

  5. Give the filter a name and leave "Predefined filter" selected.

  6. Select "Exclude... traffic to the subdirectories... that are equal to" from the dropdown boxes.

  7. Type /pingdom/ into the Subdirectory field, and leave "Case Sensitive" set to "No".

This should be the result when you're done:

Filtering Pingdom in Google Analytics

Save these settings, then repoint your Pingdom checks to example.com/pingdom/any-page-or-url-that-exists, and Google Analytics will now be filtering visits to that URL. (You'll still get basic server uptime reports this way, even though Pingdom's no longer pointing to your homepage.)

If it's not Pingdom, where are those hits coming from?

You can try to find out in a number of ways:

1. Look at what network is sending you the most bounced traffic. To do that with Google Analytics, look at the Standard Reporting section for your site, then choose Audience > Technology > Network from the left-hand side. Finally, with the pie chart mode selected (the default), choose "Contribution to total... Bounces" from the drop down on the right:

Total bounces

You'll see a list of ISPs or network providers whose users have contributed to your bounce rate. Sometimes this will be a small, specific ISP that you can trace to one area or business. Armed with the name of the network provider sending you the bouncing traffic, you could choose to filter that particular network from Google Analytics as described above, but filtering by network instead of by subdomain.

Other times, the top ISP listed will be a large domestic provider who may have millions of customers, and you'll be none the wiser. At that point, Google Analytics can't help you drill down and determine which of that provider's customers are sending repeated bounced traffic (because Analytics no longer includes IP address info or other identifying data). To get that sort of data, you can either use another stats provider who does log IP addresses (like Clicky), or you can analyse your server logs, which also records access by IP address:

2. Examine your web server access logs to explore visits from known crawlers and look for repeated access from the same IP address. A web-based stats tool such as AWStats can help you with this. It's quite popular among certain web hosts and they may already have installed it for you.

Here's some sample output from AWStats. The two relevant sections for you are "Hosts" and "Robots/Spider visitors".

The hosts section may give you clues as to what IP addresses were accessing the site to cause the bounces (and you can then filter out these IP addresses in Google Analytics). The robots/spider section may give hints about any crawlers or automated scripts that are accessing your site that might be using JavaScript. If you're not sure how to read or access server logs, ask your hosting provider for help.

Got an IP address? Find out who's using it.

If you do manage to obtain an IP address from the Hosts section in AWStats or elsewhere, you can do a reverse IP lookup to determine who might be using it.

The reverse IP lookup will often give you the name of a hosting provider who you could report abuse to if you believe that the requests are malicious. Or it may give you the name of a domestic ISP who's customer is using scripted automation in an attempt to exploit your website. You can normally send abuse reports to [email protected], where example.com is the name of the hosting provider or ISP. And, sometimes, it will offer up the name of a specific company who you can contact for more information.

1
1
+50

Whether or not it is Pingdom depends on what settings you've set for the pingdom service.

If it is set to make http requests then yes, it probably is Pingdom. If it's not using http requests (only ping or other services) then it won't be Pingdom but I would assume that you are using http requests.

I don't use pingdom so I don't know if you have control over the frequency of their tests. If you can then great. If not you could switch to just use ping tests but that will not guarantee that the website is up - only that the server is up.

If you cannot decrease the frequency of pingdom's tests and the amount of traffic it is generating is problematic, then you will just have to ditch it. However, 2000 requests a month will probably not be a big problem - they'll just royally mess up your website analytical information.

You could try to get round this by isolating the Pingdom requests and looking for a pattern. Then, if you have custom analytical scripts, you could detect the pattern and remove the IP addresses that adhere to the pattern from your analytical results. However, you might decide this isn't worth the hassle and just bear in mind the amount of false hits bieng generated by Pingdom every month.

I hope this helps.

1
  • Thanks, I have spoken to Pingdom and they believe it has nothing to do with them. I have another bit of tracking software called leadforensics that works out company name based on IP and it seems to think it's a company called NCC Group (nccgroup.com). I'm currently discussing this with their technical departments. Commented Dec 6, 2012 at 9:08
1

There's a good article on how to identify and filter out bot traffic to your website Pingdom included here's the article http://www.blastam.com/blog/index.php/2012/06/block-web-monitoring-bots-in-google-analytics/ too long to copy here

Did you know that if you are using a service like Keynote to monitor your website performance, that it is impacting your web metrics? Services such as Keynote, Gomez, AlertSite, Pingdom, and many others use a real web browser to visit your site repeatedly throughout the day in order to measure load-time performance. They are loading your site from multiple locations throughout the world with real browsers that execute the Google Analytics tracking javascript.

1

It's probably Site Confidence aka NCC Group.

For our site, a bit of investigation with GA filters revealed all the mystery hits from Edinburgh and Stamford were from a PC with Windows 7, IE8 but at 800x600 resolution which looked somewhat fishy.

We checked through our logs for any hits with IE 8 and quickly found a pattern of hits every 5 minutes; reverse DNS on those IPs and they're from Site Confidence - which makes a lot of sense, since we asked them to do it.

-2

This probably goes without saying, but it's QUITE likely this is undetected automated activity. Any activity that skews that heavily (100% bounce rate) is quite likely to be bot behavior that somehow got through GA's bot filters.

If you can, slice those visits by browser / agent; if they're all a single agent (which I'd guess), that would reinforce the odds of automated activity.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.