Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 0 down vote favorite

I've been running a number of A/B tests using Google Content Experiments, and have had the same problem across all the tests I've run in the last couple weeks. One variant (the original page) receives about 2.5 times as many visitors as the other.

The pages I'm testing receive about 40,000 visits a week, and I've run single tests for over a week, so it's astronomically unlikely that this imbalance is just a statistical anomaly.

Any ideas on why this imbalance might be occurring?

EDIT: Turns out this is a "feature" (per obscure Google documentation). My question, then, is how can I turn this feature off?

share|improve this question
I don't understand the question. Maybe some information is left out? The Google tool you apperently use ("Multi-armed bandit experiments") is designed shift traffic towards the most succesful page. Since traffic imbalance is the whole point, you can't "turn it off". – Gisle Hannemyr Jan 7 at 15:01
I'm using Google Content Experiments (Google's platform for A/B testing). I do not want the tool to automatically shift traffic to the pages it thinks is successful, though. (Because of the site's low conversion rate, I'd like to get more data before trusting one page over another---at present, the tool is making decisions based on fewer than 10 conversions.) Instead, I want to run the experiment until it reaches my own pre-determined confidence levels. – s3cur3 Jan 7 at 15:27
2  
On the page Overview of Content Experiments, it says: "Content Experiments is a somewhat different approach from either standard A/B or multivariate testing." I.e. according to Google, it is not a platform for standard A/B testing. – Gisle Hannemyr Jan 7 at 15:33

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.