Cookie-based A/B testing on same URL - SEO impact?

Question

we'll run a domain-wide A/B test comparing our old design with our new responsive design. Though the content itself remains almost unchanged, the code structure and especially the design differ much.

For the A/B test implementation we use VWO. Half of the users get the new design displayed, half the old one - by random. With the first site load the user gets a cookie which fixes him to one group.

There's no difference between the URLs of the old design and the new one. Furthermore, it's not possible to change that due to technical reasons and the time table.

For SEO, that means the googlebot will get the old and the new design randomly. And without accepting any cookies, the bot will delivered one of the types randomly EACH time it fetches a site. Will that harm our rankings? Is there a possibility to keep the bot out of one design type without cloaking?

The test will run about four weeks and after that the responsive design will be standard and delivered to 100% of the visitors.

Thanks a lot!

Best, Thomas

Answer 1

Google has written a blog post at https://webmasters.googleblog.com/2012/08/website-testing-google-search.html on the best way to handle indexing with A/B testing and multi-variant testing such as what you are doing. The first thing to note in the article is that cloaking is still not allowed even during testing. It doesn't look as though any of their options are going to work for you as they are all based on URL based a/b testing. I addition Google also states in the article to test for only the necessary amount of time then push the changes into production as if Google detects needlessly long testing it could be classified as cloaking, especially if a large proportion of your users are being shown one variant and a small number the other variant.

Having said that if the content is remaining the same and only the design and responsive layout is changing between the two variants you shouldn't have any SERP issues from running the tests. While Google will see design A and sometimes design B as long as the test isn't running for too long and definately not constantly you shouldn't have an issue.