A/B testing a site redesign without Google penalties

Question

I am redesigning our job board website to be responsive.

Rather than just replace the existing site with the responsive site, I want to A/B test the new responsive site alongside the existing site so we compare various metrics side by side.

Half of all new visitors to the site will receive a cookie that triggers the responsive version of the site from then on. Existing visitors will remain on the old version of the site.

But I am concerned that Google will penalize us for showing different content seemingly randomly (the crawler will have a 50/50 chance of seeing the responsive version of the site). The content won't be massively different but there will be significant differences; new sections, things removed etc.

Because I'm using a cookie to change the rendered HTML, the URLs will be the same for either version. In fact the URLs need to be the same otherwise lots of reports and tracking will go awry.

Because the URLs of each A/B test are the same I can't use the rel attribute to tell Google that the non-responsive version of this page is the canonical version.

Is there a correct way to handle this? Is it simply a bad idea to A/B test a site redesign?

Answer 1

You got some nice experiment going over on there :)

I have never thought of this.

I would do it a little different, I would add a pop-up (a bottom bar) informing new users that they can see the new version of the website, if they click it, the cookie will be triggered and the new css loaded.

If you just change the design, not the content and url structures, you should be fine, although I would suggest to keep an eye on the google bot crawl rate after you launch.

Answer 2

Please DO A/B test the design however, use Google's analytic tool, "Experiments", to do this.

https://developers.google.com/analytics/solutions/experiments-feature-reference

I have used this several times, with good success and never experienced a penalty as a result. Also be sure to use the mobile site check tools from google as well as the "mobility issues" in your WMT account during this process.

DO NOT cloak content in any other manner, such as what you seem to be trying to do. I dont have enough points to respond to others reply's but be careful here.

Answer 3

You could use this to determine which site to show: (pseudocode)

if (random() > 0.5 || isBot() == true){
    //show responsive site (to 50% of users and search bots)
    //and set any cookies here if you need to
} else {
    //show old site (to the other 50% of users)
}

None of the main server-side programming languages have a built in search crawler detection, but they are quite easy to find online, for example, strstr(strtolower($_SERVER['HTTP_USER_AGENT']), 'googlebot') returns true if the request is the Google crawler in PHP. There seems to be some packages on npm too.

With this method, search engines will see the new site (which they will probably rank higher), and will always see this version. And this way, you can complete the A/B tests properly with no bias.