# Site too large to officially use Google Analytics?

We just got this email from the Google Analytics team:

We love that you love our product and use it as much as you do. We have observed however, that a website you are tracking with Google Analytics is sending over 1 million hits per day to Google Analytics servers. This is well above the "5 million pageviews per month per account" limit specified in the Google Analytics Terms of Service.

Processing this amount of data multiple times a day takes up valuable resources that enable us to continue to develop the product for all Google Analytics users.

As such, starting August 23rd, 2010, the metrics in your reports will be updated once a day, as opposed to multiple times during the course of the day. You will continue to receive all the reports and features in Google Analytics as usual. The only change will be that data for a given day will appear the following day.

We trust you understand the reasons for this change.

I totally respect this decision, and I think it's very generous to not kick us out.

But how do we do this the right way -- what's the official, blessed Google way to use Google Analytics if you're a "whale" website with lots of hits per day?

Or, are there other analytics services that would be more appropriate for very large websites?

-
Looks like we now have a definition of a successful site! –  ChrisF Aug 11 '10 at 20:51
Related SO discussion: stackoverflow.com/questions/927113/… –  BenV Aug 11 '10 at 21:20
@Kinopiko - every site within the Stack Exchange network features this, including this one, just look at the top left of the browser window: within the bar you'll find the Stack Exchange logo - until yesterday it used to be a link to stackexchange.com only, but as of today (I think) it has been enriched with a JavaScript drop down menu featuring the content of stackexchange.com inline. –  Steffen Opel Aug 12 '10 at 11:13
+1 for google repeating themselves. Processing this amount of data multiple times a day takes up valuable resources that enable us to continue to develop the product for all Google Analytics users. I guess they wanted to make that clear. –  Talvi Watia Sep 17 '10 at 7:37
"I think it's very generous to not kick us out." In a regular Google fashion, there's no generosity here. You're not the primary consumer of your Google Analytics data, they are. –  LeakyCode Dec 17 '10 at 12:49

Stackoverflow.com does blow well past that limit doesn't it! Fortunately, per paragraph 2 of the terms of service...

FEES AND SERVICES . Subject to Section 15 herein, the Service is provided without charge to You for up to 5 million pageviews per month per account, and if You have an active Adwords campaign in good standing, the Service is provided without charge to You without a pageview limitation.

... as long as you have an "active Adwords campaign in good standing", then the service is provided "without charge to You without a pageview limitation".

EDIT:

As of October 2011, the terms of service have been updated such that the page view limit was raised from 5 million to 10 million page views per month HOWEVER the AdWords exemption has been removed:

FEES AND SERVICES . Subject to Section 15 herein, the Service is provided without charge to You for up to 10 million pageviews per month per account.

Consistent with this answer here it is not exactly clear what the intent is for sites that exceed the limit and if/how the limit is really enforced.

-
There's some more information at google.com/support/analytics/bin/… : "Google Analytics currently defines an active AdWords account as an AdWords account that has at least one active and running Campaign, with a minimum budget of $1 per day (or the equivalent amount in a non-U.S. currency)." – John Mueller Aug 12 '10 at 14:29 ...if everyone is scrutinizing the T&Cs for technicalities, you may want to check for the dreaded Terms and Conditions are subject to change without notice.. if so, yer up the creek. – Talvi Watia Sep 17 '10 at 7:54 So sign up for an AdWords account and send$1/day's worth of traffic to Experts Exchange. You get more analytics, and you allay concerns of antitrust litigation at the same time. :) –  John Oct 14 '10 at 20:51
@John: Nah, that'll just raise fears of collusion by the 2 biggest technical Q&A sites on the web. Besides, EE got a lot less cool after they added that dash in their URL... –  Lèse majesté Oct 14 '10 at 21:26
The new limit is now 10 million pageviews per month, with no "active adwords account" exemption. webmasters.stackexchange.com/questions/2141/… –  Yahel Oct 28 '11 at 15:16

I think that for being free, getting data updated once a day is just fine. I don't even know why with such large numbers you need more then that. Did you really lose anything?

If you really want alternatives, I suggest you try http://piwik.org/. I have no idea if it can take the load, but since it is up to you to install and configure, I guess all is possible (only questions is, how much work?)

Other hosted option is http://chartbeat.com/ which since you actually pay for the service, I don't think they will have a problem accepting your money.

Update: based on comments, http://www.getclicky.com/ is not an option.

-
Can't use Clicky either. "Please note, we cannot currently accept any sites that receive more than 500,000 daily page views (15,000,000 monthly)." - getclicky.com/user/register –  James Avery Aug 11 '10 at 23:03
Yes, Piwik is great :) –  hvtuananh Aug 12 '10 at 9:39
I started to use piwik. It is very easy to install, took me 5 minutes including configuring the first site. Drupal even has a package for it to support the collecting of data as well as showing some diagrams. And the best thing.. it is open source, so anyone can improve it and submit the improvements back to the community. It also has a plugin architecture which makes it easy to add features in form of plugins. Also piwik seems far more accurate than google-analytics, since I have the feeling a lot of people block google-analytics, which is not as easy with piwik. –  txwikinger Sep 17 '10 at 13:28
+1 for Piwik, I'd go for it –  dag729 Nov 28 '10 at 12:25
@UpTheCreek because piwik and google analytics give different results for me, with google analytics consistently being lower. –  txwikinger Dec 1 '10 at 14:08

Or just check results once a day. Is your site/service dependent on reports multiple times a day?

-
Fair point, but that is a concession on behalf of Google and technically such a situation is one where the site operator is still not in compliance with the TOS. –  Tall Jeff Aug 11 '10 at 21:21
right, mainly we want to "not be jerks" and try to follow the rules here. –  Jeff Atwood Aug 11 '10 at 22:09

If you aren't using AdWords, it may be worth contacting a Google representative to see if they're willing to make a special arrangement for you. If they're not, I'd be shocked.

-
I would be too, especially considering that it is SO. –  Tim Post Aug 12 '10 at 10:33
In your email correspondence, be sure and mention "Jon Skeet". –  GregD Dec 20 '10 at 15:32

Google Analytics is actually based on Urchin, which you can install and host yourself on your server (cross-platform). The interface looks similar to GA but it may not be as polished. It's also very expensive - nearly $3,000 for a license! If Google is still allowing you to track every hit then there is probably no reason to switch - do you really need to check data up to the minute? - Urchin 6 costs$2,995 for a license. Urchin 7, on the other hand, costs a whopping $9,995 for a single license! – Hello71 Oct 22 '10 at 19:23 Yeah, probably half of their old customer base has switched to google analytics for free ;) – UpTheCreek Dec 1 '10 at 12:40 "Urchin WebAnalytics Software is discontinued. Sales will end on March 28th, 2012" – Refineo Apr 2 '12 at 21:19 We run Piwik for a client with about 3 million page views per month. The mailing list has faster response times then Google customer support! With a bit of hacking it should be possible to run piwik on multiple machines. The other solution is to only run Google Analytics as a sampling tool. For example you only log 1 out of every 10 visitors to the site. This is what some large sites do with some of the paid for Analytical tools to save on license fee's. - Wait - Google has customer support? – Lance Kidwell Aug 12 '10 at 8:16 @lance evidently the generic email form letter kind... – Talvi Watia Sep 17 '10 at 7:49 The limit documented in the Google Analytics Terms of Service has been raised to 10 million pageviews per month, with no "active Adwords account" exemption. 1. FEES AND SERVICES . Subject to Section 15 herein, the Service is provided without charge to You for up to 10 million pageviews per month per account. It's not clear how actively this will be enforced. For sites looking to exceed this cap, Google Analytics is now offering Google Analytics Premium, an enterprise-focused paid version of Google Analytics. - According to the information document about the premium service note on the last page this service costs US$150,000 per year!! –  Tall Jeff Nov 7 '11 at 14:30
@Tall Jeff: But to put that into perspective, a site generating 10M pageviews per month can also, in theory, serve out 10M ad impressions per month. I've seen average CPM in the Tech industry cited as as high as $15. Meaning putting a single ad on each page served would net you$150K per month. At over 1M hits per day, SE could be making over $450k in ad revenue per month. I don't know that SE is making that much (after all, in this demographic, a large percentage of us are liable to have AdBlock Plus installed), but it does make Google's pricing easier to understand. – Lèse majesté Feb 13 '13 at 10:14 @Lèsemajesté - First, I think your assumption on average CPM's is very high. Certain sites certainly get that, or more, but across all sold impressions in the "tech" space, I believe the true average CPM is closer to$1.50 than it is to $15.00. Secondly, I don't see a connection between what the inventory can be sold for and what a service provider could expect to charge to simply track that inventory. Your argument would imply that a personal finance program that tracks the balance of my checking account should be priced based on how much money is flowing through my checking account. – Tall Jeff Feb 13 '13 at 16:30 @Tall: First off, that's not an assumption, that's a figure from an Adify report. For highly targeted demographics and especially more expensive ad units, a$15 CPM is not unrealistic. Secondly, tier pricing based on estimated revenue level is not that uncommon. A lot of companies out there segment their products/services into individual, SMB, and enterprise versions because these customer segments have different levels of need and budgets. And it's silly to think that a Fortune 500 country is going to spend the same amount of money on accounting software/services as you are. –  Lèse majesté Feb 13 '13 at 18:21

perhaps you should take a look at the high end analytic tools like Omniture, Webtrends, Coremetrics, Webtrekk, who are specialised in dealing with high traffic pages and allow much more customised tracking and evaluation. They cost quite some money though, but if you are that big, it's definitely an investment to take into account. Most big sites use one of these vendors and sometimes google analytics as a control.

Disclaimer: I am actually working for Webtrekk.

-

You could also use Google Analytics's own sampling functionality, which will sample traffic by user cookie and yield consistent results in the reports-- you just need to scale up the numbers you see in Analytics by dividing by your sampling rate.

-

That's fascinating stuff. My website has 10 million page views a month and I have never heard from Google. However, I've noticed a few months ago when I go to Google Analytics in the morning the numbers from the previous day will sometimes be a lot less then they should be. If I go back later in the day, then the data will be updated and I can stop freaking out at my loss of 10K visitors.

I've have recently started tracking events with Google Analytics on a much larger scale (about 60K events a day) and I wonder if all that data processing effects the speed with which my site stats are updated.

-
Just read the Google Adwords exception...I have an active Google Adwords account although I've never spent a penny of my own it's always been funded by Google's coupons, seems like a really easy way around the problem –  donaldthe Aug 12 '10 at 1:53

Google Analytics supports sampling data collection as standard now.

With this kind of volumes, a real WA solution should be considered. The Urchin7 license (really, don't go to Urchin6, I speak from experience) may be \$10k, but it's a one time fee. With many products

-

I have always heard good things about Mint: http://www.haveamint.com/

It not a hosted service so you can throw it on your own servers.

-
Mint runs on PHP and MySQL, SE sites are on the .NET stack. –  Grant Palin Aug 12 '10 at 0:52
I don't see how that really matters, it's just going to use JavaScript to track hits/visitors and I am pretty sure the SO guys know how to run a linux box. –  James Avery Aug 13 '10 at 3:33

Suggestion:

Only use analytics for conversion-related pages and metrics. For example, tracking signup conversion %, actual users referred from adwords hits, non-members, and referrals:from:other:sites. (the last can help PageRank, I have no clue why...)

Perhaps:

What you are really looking at is a two-part solution. Rather than entirely replacing google analytics, you reduce the volume by the method above and also either use another 3rd-party analytics tool or create a server-sided one.

It really is a matter of what analytic data is important to you, and do you really need Google to help you with that?

-

I've heard good things about StatsCounter, even when specifically compared to Google Analytics. Peter-Paul Koch from Quirksmode.org likes it because it's supposedly got very good browser detection accuracy

-

What about Open Web Analytics? It's free and looks googlish. Might be worth a try. It says here that it does have mouse tracking and heatclick maps. However you have to host it but it has no Data/Logging Limits.

-