I've been running a number of A/B tests using Google Content Experiments, and have had the same problem across all the tests I've run in the last couple weeks. One variant (the original page) receives about 2.5 times as many visitors as the other.

The pages I'm testing receive about 40,000 visits a week, and I've run single tests for over a week, so it's astronomically unlikely that this imbalance is just a statistical anomaly.

Any ideas on why this imbalance might be occurring?

EDIT: Turns out this is a "feature" (per obscure Google documentation). My question, then, is how can I turn this feature off?

  • I don't understand the question. Maybe some information is left out? The Google tool you apperently use ("Multi-armed bandit experiments") is designed shift traffic towards the most succesful page. Since traffic imbalance is the whole point, you can't "turn it off". Jan 7, 2013 at 15:01
  • I'm using Google Content Experiments (Google's platform for A/B testing). I do not want the tool to automatically shift traffic to the pages it thinks is successful, though. (Because of the site's low conversion rate, I'd like to get more data before trusting one page over another---at present, the tool is making decisions based on fewer than 10 conversions.) Instead, I want to run the experiment until it reaches my own pre-determined confidence levels.
    – s3cur3
    Jan 7, 2013 at 15:27
  • 2
    On the page Overview of Content Experiments, it says: "Content Experiments is a somewhat different approach from either standard A/B or multivariate testing." I.e. according to Google, it is not a platform for standard A/B testing. Jan 7, 2013 at 15:33

1 Answer 1


the imbalance is probably well explained by the "multi-armed bandit" approach that Google's statistical engines take in the back-end. I have noticed this with other users too. However, have you tried setting the option "Distribute traffic evenly across all variations" to ON? Google does the auto splitting and auto allocation when you don't explicitly say to distribute the traffic.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.