0

I am currently working on a website, and we have just started an A/B test for revenue. I can not use exact details, but my question outline is as follows. Please note I am a developer, I am purely posting this query out of interest for myself. It may be a fundamental question which marketers would know by default, so please don't assume I know what I am talking about if it turns out to be simple.

We have a website (for example) which sells cats, you go to the website and can select the number of cats you want (they all look the same, are as cute as each other). They cost £5 each. There is a dropdown on the page which allows you to select a hat for the cat. The default blue hat is free, but a snazzy gold hat costs £3 extra. So if you pick a gold hat, the total costs are £8 per cat.

When you get to the page, four cats are selected by default (including a free blue hat each). So the total starting displayed example price is £20 (4 x cats, 4x free blue hats), you can amend this as you want.

When the user makes an order we are using tracking this revenue using e-commerce tracking in GA.

OK, so we now have created a 'B' page from the above with the gold hats auto-selected (to A/B test). So when the user gets to the page, the default starting displayed example price is £32 (4 x cats, 4x £3 gold hats). The user can again amend this (even switching the dropdown back to free blue hats).

OK, my query is, if we are running this test using the total revenue. Is this fair? As the price for the B is higher. We may make more money from 'B' but how would we know if we have lost more sales because people thought it too expensive (without realising they could opt for free hats initially). Do the lost sales matter? Or will it all work out in the end?

Any recommendations welcome.

Edit: Edit2 As per comments, I have posted the last edit as the answer.

  • It would be better to post your edit as an answer. Can you do so, please? – Stephen Ostermiller Jul 13 '16 at 10:13
0

OK after looking at the stats from GA, it looks like this is actually taken into account. GA takes the total revenue from all the A/B sessions and then works out and average from all the sessions for comparison together. This includes any sessions which ended with a £0 transaction, i.e. no sale. So by default, the comparable stats take into account possible lost earnings of a higher cost of one of the experiments.

Hopefully this is correct and will help another user with the same issue. If its not please correct me below.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.