I'm running an A/B test on a website that uses a goal completion as the primary objective (Sign up form). However, I've noticed that in the goals overview tab the number of conversions is higher than the number of conversions recorded in the experiment results (over the same period). Since the experiment is using the same goal shouldn't the number match exactly? What could be causing this?
Web tracking is inaccurate. Stats from different trackers almost never match exactly. There could be many reasons why:
- One of the trackers is removing duplicates
- Bot traffic often inflates tracker numbers, one of the two may be doing a better job of filtering it out
- They have different definitions of sessions but record only one conversion per session.
- One of the trackers takes longer to load, so some users aren't hitting it because the leave the page before the tracking happens
- Users more commonly block one of the trackers than the other with browser privacy extensions or ad blockers.
- One of the trackers servers went down for a bit, or lost some data
- The tracking happen at different stages and users get lost in between. For example if you run advertising, the number of clicks the advertiser says they sent is almost always higher than the number you observe on your server. Some of that could be advertiser fraud, but users often click on something and then use the back button before it finishes loading.