Use Google Analytics to A/B testing with the same URL

Question

I want to know if it's possible to perform an A/B test on different versions of the same URL using Google Analytics.

So far, I have performed A/B tests on different URLs like xxx1.html and xxx2.html then compare both.

But what I want to do is to do an A/B test with the same URL but with the current version and a forthcoming version. Is that possible?

Answer 1

You can do this using the content API. Its a bit confusing to set up, but here are some hints following this guide: https://developers.google.com/analytics/solutions/experiments-client-side

First you need a javascript/jQuery script in the head(ish) area of your template. Each function() within expVers[] is a variant of the experiment. You can do as many as you like, however 2 or 3 seems to perform the best, with one of the 3 quickly being weighted by Google (few days). This is a pseudo code for adding either red or orange class to btn's:

<!-- Include the content API snippet above this. The Google hosted snippet can fail for some reasons so I paste it in. -->

<script type="text/javascript">
    // Set variants as indices
    var getIndex = cxApi.chooseVariation(),
        expVers = [
            function() {
                log = 'Variant 1';
                $('.btn').addClass('red');
            },
            function() {
                log = 'Variant 2';
                $('.btn').addClass('orange');
            }
        ];
</script>

So those are defined, now we need to actually trigger a change for the user after the DOM is loaded. This is a bit simpler, put it below like in your footer:

<script type="text/javascript">
    // Trigger and show variant to user
    $(function() {
        expVers[getIndex]();
        console.log(log);
    });
</script>

Now there are 3 things to consider that the guide doesn't really talk about.

1) You can't delete past experiments [without using the analytics API]. So, make a new property to test with instead of doing it on your main property. You don't want to junk up your main property with experiments that were for testing, failed, or whatever.

2) Make sure your IP is being ignored by a Google Analytics filter else your testing may wonk results. These experiments are cookie based. So when you test, you can test switching variants or weighting by deleting your GA cookie and refreshing. Or if you use this method, you can do query strings to "force" a variant to show while testing. Also, in the snippets above, there is logging. This is important for various reasons such as more ambiguous or "under the hood" changes that aren't necessarily easy to see. Press F12 (chrome inspector) and hit the console tab. You will see what variant has fired, or any other info you want to add to logs.

3) Since this experiment is client side, there may/will be a FOUC situation for a moment as the page first loads for at least 1 variant pool. What I mean by this is say you are testing 3 button colors. They are blue by default, and you are running an experiment to test red and orange variants too. So a user hits, sees blue buttons for a moment, then JS kicks in and changes them to either red or orange. This happens because of the delay before DOM is ready (JS doesn't run before DOM, per-say).

There are various ways to handle this, such as making buttons invisible with CSS or making a loading overlay, until JS is loaded and makes visible. Each instance/site/theme may require a different way of "glossing" this to avoid FOUC or any flickering, etc. If it's just a minor change, perhaps you can live with it....but a FOUC/flicker with bigger changes can affect the results of your experiment slightly. Think of a flickering logo for example. Or a user on a very slow connection having a huge delay before JS kicks in. Good luck experimenting!

Answer 2

Yes.

One way is to use the content experiments API: https://developers.google.com/analytics/solutions/experiments-client-side. Essentially what you'll be doing is using Javascript to create the variations and letting the API tell the page which to display. If you're comfortable with JS, this seems to be the way you want to go.

Of course, there are other services like Optimizely, VWO, Unbounce, etc. that can handle most of that process so you don't need to write the JS. Not every one will likely work for your specific situation, though.

Answer 3

An alternate method

In my case, the server-side application handled serving the A/B and they were separate templates. I'd prefer not to use JavaScript to manipulate the DOM for my changes... that feels limited and also suffers from the FOUC situation as mentioned in the selected answer.

What we did was sent a custom event to Google Analytics after sending the 'pageview'.

First, send a custom event to Google Analytics

// For an "A" visitor:
ga('send', 'pageview');
ga('send', 'event', 'Visitor Variant', 'A', '', { nonInteraction: true });


// and for a "B" visitor:
ga('send', 'pageview');
ga('send', 'event', 'Visitor Variant', 'B', '', { nonInteraction: true });

The "nonInteraction" flag is necessary or this event will count as a second pageview, artificially reducing your bounce rate to 0% for all visitors.

Then, compare A/B in Google Analytics via segments

For reporting, simply create segments in Google Analytics using the "Event Category" of "Visitor Variant" and "Event Action" of "A" or "B". The labels of "Visitor Variant", "A", and "B" are all custom and you can change them to whatever you want, provided that they match in the JavaScript above and in Google Analytics.

Disclaimer

This is probably not the canonical usage of Google Analytics custom events, but it did the trick for me.