Tag Info

Hot answers tagged

9

Read a great introduction to A/B testing on the smashingmagazine.com. The piece includes links to multiple A/B testing tools.


8

The Visual Website Optimizer blog has an A/B test duration calculator designed to answer your question. It's published as an Excel spreadsheet, but don't let that put you off. Enter your original conversion rate, the difference you're hoping to detect, the number of variations you're testing, and your average daily traffic, and it spits out the number of ...


7

One of the key things to understand when doing A/B testing is the statistical significance of your results. Some good posts on this can be found here: Easy statistics for AdWords A/B testing, and hamsters Statistical significance & other A/B test pitfalls Significance of Statistically Significant Results in A/B Testing


5

I hope I'm understanding your situation correctly. If I'm not, leave a comment and I'll clean it up a bit. In your situation, I'm not sure how well A/B testing will work - I'm concerned with being able to get usable results. There are two problems to overcome - getting statistically valid results and then understanding those results. The first problem that ...


3

You could do it on the same page by passing a parameter to the page, so http://www.yourwebsite.com/page.html?version=2 and http://www.yourwebsite.com/page.html Set the colour of the button based on the parameter sent to the page. Google Experiments work by redirecting anyone who lands on http://www.yourwebsite.com/page.html to any URL of your choice ( ...


3

MixPanel is reasonably priced, has an excellent feature set and has emailed reports.


3

I would be reluctant to do it. I assume you're trying to optimize your conversions. Here's why: You're breaking a well known UI pattern. It will cause confusion and unease to see something "new" like this. If the field is a password field (i.e. masked characters) there will be a decent number of people that will type their password wrong, or will freak ...


3

One option would be to set up multiple channels for each site, and then use those channels for your split testing. Earnings on a particular page in analytics could be somehow done if you add a parameter to one version of the page depending on your frameowrk and how you are modifying the pages themselves for the test. I am not quite sure how the different ...


3

Someone in the client's office read an article about how all the large agencies do A/B testing of concepts prior to going live....so it must be important for them to do too! All well and good if you're being paid large agency money for large agency results on a large agency site... unfortunately, as Kris and Thomas Owens have noted, statistical ...


3

A/B testing has its place but it requires some things to hold. Most importantly you need a clear success metric. I.e. it must be straightforward to judge which performed better A or B. Primarily this holds if your site is oriented at a specific commercial goal (e.g. get people to place purchase orders, get people to click on ad links etc.). You also need ...


3

This mighty list has a selection of A/B & MVT tools: http://www.whichmvt.com/ No relationship with either the list compilers or any of the listed vendors (we have out own in-house MVT tool).


3

Google website optimizer is free and used by a lot of people I've not tested it with Drupal but here's a link with step by step instructions so I would think you would have no problems http://www.drupalseobook.com/news/google-website-optimizer-with-drupal. (since it's free it's easy enough to test it first). A paid option which I use and highly recommend is ...


2

Instead of doing something with your robots.txt which doesn't always do what you want, the better option is to just include a <link rel='canonical' href='XYZ' /> tag in all version of the page you are A/B testing and just make XYZ equal to the original/master version of the page. That way you know for sure that Google is crawling at least one page. ...


2

In the past, I've done limited A/B testing and just used a modulus operator on the page to split on a random number. Then I showed one or the other items, and tracked in the log file... Not ideal I know. I did see this though that may give you more info down the path you're looking for... ...


2

Look for the _utmx cookie for the site in question. From Google You can use Google Analytics with Google Website Optimizer (GWO), which is a tool that helps determine the most effective design for your site. When a website optimizer script executes on your page, a _utmx cookie is written to the browser and its value is sent to Google Analytics. See the ...


1

A simple way to show one of two options is using PHP: if ( rand(1, 2) == 1 ) { // show test feature // Google Analytics tag or event } else { // show default feature // Google Analytics tag or event } This will show each option evenly over time. Personally I would trigger Analytics events to track the results, but there are probably many ...


1

I do not know of Google Tag Manager per se, but a split test is generally an equivalent to a hidden 301. So behind the scene, when someone goes to your page, say http://www.example.com/GTMPage/, then the server decides to show data A or data B. A would be your existing page. B would be a refined version of the page. In effect, in this case you send the ...


1

A/B testing for SEO can't use Google Experiments or other methods that change pages on a per-user basis. You need to make sure that Googlebot and users each see a set of pages from group A and another set from group B. I generally pick half the pages on my site to change, and half to leave "as is". This only works if you have a large number of similar ...


1

Set up a Google Analytics account and implement the code on your website. Then cross-check that it's tracking your site correctly. If done right, head over to "Experiments": Enter the URL of the page you'd like to run your test on: After which you are presented with this: Now you simply add another variant of your cart2.php page there and let Google ...


1

Searching around I'm going to give 302 redirect a try. Seems to be better supported and not have a penalty attached.


1

the imbalance is probably well explained by the "multi-armed bandit" approach that Google's statistical engines take in the back-end. I have noticed this with other users too. However, have you tried setting the option "Distribute traffic evenly across all variations" to ON? Google does the auto splitting and auto allocation when you don't explicitly say to ...


1

Why wouldn't you use Google Website Optimizer? On dynamic pages such as big ecommerce sites with template systems and SEF URL's I use the multivariate option with a single variation rather than the A/B test which forces a redirect to a static page. I've also ran multivariate tests with 8 variations of a single pages content. I would use GWO as Google will ...


1

What you ask is a cost problem in your UX evaluation. As you know, cost = frequency (how many times a problem occurs) x gravity (the cost of the problem for you and the user). You can have a first evaluation of the cost for the repeated field : it always append you can measure the evident cost of this field, ie the cost you can mesure without a test ...


1

WebsiteOptimiser redirects to your a/b test cases, so Google itself should be ok, but... I add exclusion rules to robots.txt, as it is possible (albeit not that likely) that someone goes to your site whilst you're testing, and reposts the link ./index_b.php to their blog etc, from which point your split test URL is likely to be crawled. So much depends on ...


1

Google Website Optimizer sets the ___utmx cookie - your site can respond to this cookie (either via server-side script or via GWO's default Javascript) Using Javascript to display text content is suboptimal given that search engine spiders will miss the content: I would recommend serving your default "A" content for users with ___utmx variant A and for ...


1

Not enough experiments or divergence in the results to be statistically significant yet. Try doing a Google search on AB Testing confidence and work out what sort of confidence level you're at with your experiment. This will give you a much better idea of whether your results are significant or whether you should just let the experiment run longer. ...


1

This answer is Drupal specific, as I believe your question was project/module based. We've used Multivariate to do a range of multivariate or AB testing. I know there is a Webform AB test module as well, but the options are somewhat more limited. While this question was over a year ago and you've no doubt moved on - hopefully someone else can benefit from ...


1

We visual website optimizer http://visualwebsiteoptimizer.com and there is a drupal module for that too, it's made us convert a good deal better, can't recommend it enough.


1

If you're looking for specific A/B testing tools, A/Bingo is a great plugin for Ruby on Rails sites looking to do A/B testing. And since its built into the framework, you don't have to physically build two pages to test against each other. Instead, you can easily just create two buttons or images and it will handle the rest.


1

Here's a great slidedeck: A/B Testing Framework Design by Patrick McKenzie. Wikipedia has a good definitional article on A/B Testing: A/B testing or bucket testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates. A classic direct ...



Only top voted, non community-wiki answers of a minimum length are eligible