# Sample size and statistical significance in Google Analytics

I have been asked to compile a report into dropout rates during checkout for a global webstore

I have used a sample size over one month as my sample because:

1. google analytics slows to a crawl over larger sample sizes and makes much of the analysis agonisingly slow
2. I believe it to be statistically significant and a representative sample

My client has asked me why I didn't use yearly figures and wants proof that one month of data is 'statistically significant'.

Am I right in thinking that I need to compare the standard deviation of my monthly sample to the yearly sample and ensure that the deviation is under a certain %age?

Question: how do I prove one month of Google Analytics data is representative to one year worth of data?

Stats:

• 90k unique views/month
• ~1.1m per year.
-

(I'm not an expert in statistics.)

One month's data is almost certainly not representative, for a couple of reasons.

1. Most sites have seasonal trends (both monthly and weekly). Picking a really light month or a really heavy month will skew your results.

2. You'll probably need a sample size of more than 1/12 to get results worth betting business on.

It would seem to make more sense to take a random sample of 'n' days within a single year, where 'n' is chosen to give you the confidence level and margin of error you feel you can justify.

There are online tools that can help you choose a sample size. I linked to the first one I found. I don't have any specific experience with it or the company that hosts it.

Am I right in thinking that I need to compare the standard deviation of my monthly sample to the yearly sample and ensure that the deviation is under a certain %age?

I don't think so, but I could easily be wrong. The standard deviation of a very small sample might match your population standard deviation without the sample size being representative.

I need to brush up on my stats.

-

`Statistically significant` is a terms used in Maths that means your data is indicative of a certain event and that you have used enough data to prove it.

For example, if 5 people came into your shop and 1 of them bought something, you can't then advertise that 1 in 5 people spend money in your shop.

When I was doing my A-levels, we had lookup tables for different distributions and sample sizes where with some calculations you could calculate how significant your results were. Unfortunately my brain gives out at this point trying to remember exactly how... A Google search should throw up more info.

-
Just a little note: statistic significance it's a term used in statistics (not Maths :) ). In this case the sample is probably statistically significant, the problem is about the quality of the sample (i.e. if the traffic is seasonal then the analysis is skewed). As the site has 90k visitors in a month, and 100 visitors being about 0.1% of the total could not skew the results - as opposed to your example where 1 new visitor could change the stats by a long way as you said). – milo5b Aug 30 '12 at 18:02

There's usually traffic patterns for every eCommerce site. Peaking around Nov through Dec. Halloween peaks in Oct and is dead the rest of the year. Google search traffic varies month to month. Login to adwords go to the keywords tool and export to CSV and look at the search volume for some of your keywords you'll see what Google shows in the keyword tool is the average of the last year. While the CSV will show you how some months search volume goes up quite a bit compared to others.

How would you pick a single month of data to analyze when traffic and search volume can vary from one month to another?

If Google is running slow when you're trying to generate a yearly report you should drill down and create very specific custom reports for 1 years time. I've generated reports for 1 year with comparison data of the previous year and got them pretty quickly. I would imagine that your bounce rate from the checkout pages is much less data than gathering visits to your home page.

I would do what it takes to get not 1 but two years of data to compare and show them what's changed in bounce rates over the last year.

-

I would show that the traffic is not seasonal, so there should not be much difference comparing any month with each other (assuming no significant changes in the website/traffic source has been done).

Standard deviation would be one signal, but I think any descriptive statistics analysis would do it. However you should not base your demonstration solely on numbers, but behaviour as well (even if only main traits, then going into depth for the month you analysed).

-