Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [a-b-testing]

A way to test which version of a website performs better by showing one version to 50% of its visitors and another version to the remaining 50%.

1
vote
0answers
14 views

How can I track in Google Analytics a long list of all the A/B tests a user is assigned?

For our site's A/B testing requirement we need to track all the tests a visitor is experiencing. This is set to grow to around 100 potentially running concurrently in the future. We would like to ...
1
vote
0answers
15 views

Will switching to server side experiments improve performance and eliminate the bias introduced by client side experiments?

We have a server site rendering React application and currently are using client site A/B testing tool. The issue here is following: User comes to the site Server loads default version Client detects ...
1
vote
0answers
22 views

Google Optimize A/B testing freezes the homepage

I'm trying to run 4 A/B tests using Google Optimize, on the same domain. The test should exclude themselves via the targeting, however, in most cases when I access the homepage (see: test 1a, 1b below)...
1
vote
1answer
25 views

Simple A/B testing tracking question for GA

I'm wanting to do simple A/B testing for my website and curious how I can leverage the basic features of google analtyics to do this. I found a very close question that already asks what I want, but ...
1
vote
1answer
70 views

Sitewide A/B or Sanity Test

We are switching to a new Wordpress template and need some help. We will monitor if the test version improves the sites performance (CTR, Revenue) Technically we will: Split A Records by 50% for ...
0
votes
0answers
14 views

Google Analytics cohort analysis returns different data refreshing the page

WHAT ARE WE TRYING TO ACHIEVE We are using Google Analytics Cohort Analysis to AB test a feature. We generate a parameter on the client side and set 50% of users to "Yes" and 50% to "No". This ...
1
vote
1answer
32 views

Google analytics reports a higher number of conversions than the a/b test (Google optimize) does for the same period

I'm running an A/B test on a website that uses a goal completion as the primary objective (Sign up form). However, I've noticed that in the goals overview tab the number of conversions is higher than ...
1
vote
0answers
40 views

Tracking alternatives withing goal funnels

I honestly have no idea how to phrase this better. I need to take a regular goal funnel tracking and make it report on a few alternatives within one of the steps. Here's an example: back button ...
1
vote
1answer
117 views

Does SEO ranking gets affected if I run a split URL test on our product detail page (PDP) using Visual Web Optimizer (VWO)

We are trying to run a vwo test for our pdp page.I created a test in VWO where URL www.example.com/products/productid will get redirected to www.example.com/productid?isVWOVersion=true for 50% users. ...
1
vote
1answer
276 views

Event based triggers in AB testing using VWO

Is it possible to create an AB test where 100% of users see a page and only those that click a button are entered into the variation in VWO (Visual Website Optimizer)? I have asked the company but ...
4
votes
1answer
44 views

What metrics would be useful to A/B test site search

We are trying to implement new type of instant search to our e-commerce. Our present search has lots of filters like price, category, review etc. What kind of metrics would you use to analyse if it ...
1
vote
3answers
974 views

Use Google Analytics to A/B testing with the same URL

I want to know if it's possible to perform an A/B test on different versions of the same URL using Google Analytics. So far, I have performed A/B tests on different URLs like xxx1.html and xxx2.html ...
0
votes
1answer
48 views

Google Analytics A/B revenue testing when the test has a price change

I am currently working on a website, and we have just started an A/B test for revenue. I can not use exact details, but my question outline is as follows. Please note I am a developer, I am purely ...
2
votes
1answer
143 views

When should I distribute traffic evenly across all variants of an experiment?

Actually we test our trust symbols in our shop with an GA experiment. (all trust symbols vs. no trust symbols) Our last test run 16 days and declared a winner while it had only 0.26 % better ...
1
vote
0answers
28 views

Why is GA Experiment crushing my whole report?

Until yesterday, we had a clean reporting of our content urls that looked like: /men/shirts/shirt-with-extended-lines?color=12988 But then we activated an experiment and since then we just see this ...
1
vote
1answer
1k views

What is a confidence threshold of 95,0% in Google Analytics Experiments?

In Google Analytics Experiments you can Set a confidence threshold 95.0% (or 99,0% or 99,5%). The documentation says: Set a confidence threshold: The minimum confidence threshold that must be ...
1
vote
1answer
336 views

Cookie-based A/B testing on same URL - SEO impact?

we'll run a domain-wide A/B test comparing our old design with our new responsive design. Though the content itself remains almost unchanged, the code structure and especially the design differ much. ...
1
vote
1answer
516 views

How to check Google Analytics Testing behind .htaccess wall?

I am implementing Google Analytics Test and I need to check if it works before we deploy it to live. But our staging server is behind a .htaccess security. So the Google check from Analytics does not ...
2
votes
2answers
240 views

How can I test different types of buttons with Google Analytics Tests?

A few years ago I did A/B testing with Google where I put javascript into my page and it displayed two or more versions of a code snippets. So I had ONE url with e.g. different buttons (green and red ...
3
votes
3answers
175 views

A/B testing a site redesign without Google penalties

I am redesigning our job board website to be responsive. Rather than just replace the existing site with the responsive site, I want to A/B test the new responsive site alongside the existing site so ...
1
vote
3answers
395 views

Is it possible to A/B test web servers using a PHP redirect and sub domains without breaking all the SEO goodness?

We're trying to figure out if a certain type of web server architecture is better than our current web server. We need to A/B test these 2 boxes with live traffic to really get a sense of which one ...
5
votes
2answers
12k views

What does the % new session metric in GA Content Experiment refer to?

My understanding of the GA entity hierarchy model is the the profile entity is the parent of the experiment entity. If you go to the content experiment dashboard, click the site usage tab in the ...
1
vote
0answers
106 views

How are Google Analytic experiments impacted by filters?

Google Content Experiments allow you to create experiments for A/B testing two pages, which then slowly converge on the "better" page depending how the goals are met. However how is this convergence ...
1
vote
1answer
29 views

Website testing for some percent of visitors

What's the name of website testing method when the new feature is shown only for some users (not for all), for instance 20%? Are there some online services for that? I know about A/B testing but I'm ...
1
vote
1answer
1k views

Google Analytics for cookie-based A/B testing (same URL for A and B)

I have a website with share-friendly URLs like example.com/register I implemented A/B testing as a cookie because I don't want ugly unsharable URLs like example.com/registerA and example.com/...
1
vote
2answers
275 views

Does the JavaScript that Google Analytics Experiments adds slow down alternate versions, and how could I measure this?

I am using Content Experiments in Google Analytics to test three different landing pages. This uses JavaScript on the initial page to load the alternates. Would the additional JavaScript add to the ...
1
vote
1answer
206 views

Is it possible to A/B test javascript-generated element variation using GA Experiments and GA Events?

I'm trying to do A/B testing using Google Experiments on a javascript-generated variation elements. In simpler example, I'm trying to create a button using javascript that has different colors The ...
2
votes
1answer
218 views

A/B Testing a site-wide feature with Google Analytics

I am considering a major adjustment to the search-input feature on a website. This search feature appears on many pages. I do have e-commerce operating in Google Analytics. I have glanced at ...
2
votes
3answers
705 views

Is doing A/B tests using site redirection a bad practice?

I'm developing hotels websites here in Brazil. When the site is done, we do an A/B test with the old version to measure conversion and show to the hotel owner how good our site is. Due to the fact ...
1
vote
1answer
110 views

How to make a split test that reduces scripts from Google Tag Manager in a tool like VisualWebOptimizer

We have a website, where we use Google Tag Manager, in order to insert a big amount of scripts. We want to run a split test, where we reduce these scripts by 70%, which should improve load time (even ...
1
vote
0answers
43 views

Google Experiment with multi step shopping cart

I am attempting to run an experiment that tests an alternate checkout UI. For the experiment, I use the URL of the original checkout and the URL of the alternate checkout as a variation. Google ...
1
vote
2answers
92 views

Is it possible to A/B test SEO using Google Experiments?

I've read that A/B testing does not have a negative impact on SEO, but is it possible to A/B test to improve SEO? My thought is to track each visitor's referring domain and create an event when any ...
2
votes
0answers
78 views

Can calling mixpanel.track cause reattribution of conversions in Mixpanel funnels?

I am currently using MixPanel to A/B test which of two different landing page images converts better. At the beginning of each hit to the page, I do mixpanel.track('Main', {illustration_ab : '{{...
2
votes
1answer
162 views

Google Experiments - can you run across the entire site?

Is it possible to run a Google Experiment A/B test across an entire website, rather than just a single page and a variation?
1
vote
1answer
61 views

Google Analytics Content Experiments for non-simultaneous tests

I really like how Google Analytics displays the results of content experiments. However, it seems the tool only works for simultaneous tests. I'd like to use the tool without implementing the page ...
2
votes
1answer
305 views

How to conduct A/B split testing with AdSense?

Ok so I have decided to A/B test my AdSense ads. I have run a few tests, but I don't know what conclusion to draw and how to keep track of things. Some specific questions: If I have 2 test units, 1 ...
2
votes
1answer
278 views

Google Analytics Experiment — How to test shopping cart

We have: GET landing.html (then select a production) POST cart1.php (then select options) POST cart2.php (then enter billing info) POST cart3.php (will process payment and then ...
4
votes
1answer
98 views

Are 307 Temporary Redirects safe for temporary new web site tests?

Our current site is getting close to being replaced. I was looking to do a test of our new sales funnel on the new site and compare the traffic and conversion ratios for a few days. So I was planning ...
0
votes
1answer
180 views

One variant receives way more visitors than the other (Google Content Experiments)

I've been running a number of A/B tests using Google Content Experiments, and have had the same problem across all the tests I've run in the last couple weeks. One variant (the original page) receives ...
1
vote
1answer
141 views

Alternative to Google Analytics with A/B testing and PDF scheduling

Is there a hosted solution that competes with Google Analytics and has some of the heavy hitting features: A/B testing with regression analysis Conversion tracking Funnels Schedulable reports that ...
0
votes
1answer
56 views

How to know if any website is using Google Website optimizer for A/B testing?

How to know if any website is using Google Website optimizer for A/B testing? By viewing a source code. For what to look into view source which confirms that site is using Google Website optimizer
2
votes
1answer
342 views

Long pages or accordion for content?

I'm looking at how best to display content for my website. I plan to have a large number of pages of products, which will contain a large amount of information (ideally, the amount of information on ...
3
votes
3answers
365 views

Will multivariate (A/B) testing applied with 302 redirects to a subdomain affect my Google ranking?

I want to do an A B test of an entire site for a new design and UX with only slight changes in content (a big brand site that has good Google rankings for many generic keywords. My idea of ...
3
votes
2answers
179 views

Has anybody split tested one password field versus password & confirm on registrations?

I'm just curious to see if this is something really worth testing, as it will require some coding to achieve.
4
votes
1answer
405 views

If you run a split test, how long do you let it run before you conclude it is inconclusive?

When running A/B tests with Google Website Optimizer, Google tells you when a variation has "won". However, sometimes, the test runs for a long time and still there is no winner. When should one ...
3
votes
2answers
475 views

Should I disallow robots on A/B test pages?

I'm going to be setting up an A/B test using Google's WebsiteOptimizer on my homepage, and I plan to name my two test variants index_a.php and index_b.php. Should I be putting in rules to my robots....
2
votes
2answers
189 views

Weird Results A/B Test in Google Website Optimizer

I set up a test in Google Website Optimizer that has a 3 variations - original (A), B, and C. In order to further validate the results of the test, I added a variation C that is exactly the same as ...
8
votes
3answers
3k views

Whats the best module or method for A/B testing with Drupal?

I'm looking into rolling our company's static websites into Drupal. I need to ensure that I won't ruin our chances for split testing our designs and layouts. I have found several modules out there, ...
0
votes
4answers
3k views

How to A/B test two different site designs

I'm familar with A/B testing variations of a page via Google Website Optimiser but I'd like to A/B test completely different designs on a site wide basis or a collection of pages. So each visitor ...
6
votes
1answer
556 views

Has anyone used Genetify for A/B testing

I'm working on a small project that will likely run on Wordpress, I always like to run some split testing to improve conversion rates for various goals. Typically if its a small site that I either don'...