Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I'm familar with A/B testing variations of a page via Google Website Optimiser but I'd like to A/B test completely different designs on a site wide basis or a collection of pages.

So each visitor would see either design A side wide or design B site wide.

I haven't yet found a service that does this (or I haven't realised how to do it).

Can anyone help?

share|improve this question
add comment

4 Answers 4

Google Website Optimizer sets the ___utmx cookie - your site can respond to this cookie (either via server-side script or via GWO's default Javascript)

Using Javascript to display text content is suboptimal given that search engine spiders will miss the content: I would recommend serving your default "A" content for users with ___utmx variant A and for users + spiders with cookies disabled and your "B" content for users with variant B specified in the ___utmx cookie.

So long as you are not modifying the value of the ___utmx cookie after it is initially set, you should be able to use GWO to continue multivariate testing across multiple pages on your site.

share|improve this answer
    
Sure I could do this (probably via PHP I suppose) and serve different designs based on the cookie value. I'm thinking google might not like this though? Have you had any experience with this on a site that has been around a while and gets most of its traffic from Google? –  Che Jan 4 '11 at 8:51
    
@Che: If the site content is the same, Google shouldn't care--especially if you use proper semantically structured documents and the design changes are effected by simply swapping stylesheets. –  Lèse majesté Jan 4 '11 at 13:11
    
Thanks, the changes more than just CSS and consequently I feel a little nervous about this method. As @tadamson points out I'm probably looking at what I'm trying to do the wrong way. –  Che Jan 6 '11 at 8:44
add comment

There are a number of services available out there for you to do this - most notably a cool free one is:

Google Website Optimizer

http://www.google.com/websiteoptimizer

share|improve this answer
    
Thanks for the reply but I use this already, as I indicated, for A/B testing on a page by page basis. I want to test two designs applied to the entire site. –  Che Jan 4 '11 at 8:33
add comment

How about creating two different subdomains?

www.yourwebsite.com

www2.yourwebsite.com

Redirect 50% visitors randomly from www.yourwebsite.com to www2.yourwebsite.com (A better strategy is to have a pop-up that says - would you like to try our new design? See http://www.namecheap.com/ who are currently running a beta and do such a thing.) Of course if you don't want the user to know or chose, just automatically redirect them.


Also, since you want proper stats for A vs B via some third party service like google analytics, its better to have the main page www.yourwebsite.com only as an entry door till the time you are testing, and have

www1.yoursite.com

www2.yoursite.com

as two different domains. Why? Because that way, the redirection page counts will not interfere with the real stats of A vs B.


You can easily configure your webserver to have different document roots for different subdomains, so managing files is pretty clean.

share|improve this answer
    
Thanks, I hadn't thought of prompting and then redirecting. Am still reluctant to use it bacause I want to find a method to test like this on an ongoing basis. But it is a nice simple solution and would likely generate some useful data. –  Che Jan 4 '11 at 8:45
add comment

A/B testing is intended for testing specific, isolated details - basically the scientific method applied to user interaction / visual interface stuff, with all the meticulous attention to detail that implies. For broadly testing two completely different designs against each other, you won't get any of the benefits from it. You'll be able to find out if one design performs better than another, but you won't find out why, which is the whole point of A/B tests. This could be why you haven't found any services that do this - splits are irrelevant at that wide a scope, there are too many variables at play to pull any reasonable data from.

If you just want to find out which design works better, then instead of A/B testing, it might be easier and just as effective to just set the two designs up as independent sites (with equal marketing & stat tracking) and let nature take it's course. Or just swap one design for the other until you can see how each performs, or whatever sounds reasonable. Once you've figured that out, that's when A/B testing will start to get useful, analyzing & improving the individual, isolated elements of the design.

But if you're really stuck on the idea of using splits for this, you could theoretically implement it yourself server-side using libraries like PHP A/B or Swab for Python (among a dozen others for any platform), or just a few if-else statements and some stat tracking. I wouldn't recommend relying on a 3rd-party / Javascript-reliant solution for it though, it sounds too volatile - should the test provider go down, your site may go down with it. Kind of defeats the purpose.

share|improve this answer
    
Thanks, that makes a lot of sense. I've been looking at it slightly the wrong way and am using the term incorrectly. You are right, all I want to do is test which design converts better overall and I'm probably trying to use the wrong tool for this job. I'll rethink what I'm trying to do and check out PHP A/B but maybe two sites and GA is all I really need –  Che Jan 6 '11 at 8:48
    
I did the same a few years back, and it still kind of makes sense - most A/B tools will force each user to only one version of the site, making results more honest. If you can figure a way to do that with 2 separate sites, there's probably a market for it :) –  tadamson Jan 6 '11 at 9:12
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.