Google Analytics 4 withholding data due to thresholding

Question

I'm a web developer using Google Analytics. A client asked us to build a fun "personality quiz" with 4 simple questions. At the end of the quiz we'd show a description of their personality with some custom music to go along with their responses. The combinations are 4x4x4x2 = 128 possible outcomes. The client wanted us to implement Google Analytics so we could learn which answers were most popular, so we added custom events using this approach. For example:

gtag("event", "cta-select", {
    "event_category": `question-${page}`,
    "event_label": `answer-${answerID}`
});

So each time a user picks an answer, the event would track question-1, answer-3, for example. We tested this using the Realtime tracker, and the events were submitting as desired. However, after thousands of visitors, we wanted to take a look at the report, and the data is being blocked due to "thresholding":

The Learn more link takes you to this page, which says

Data thresholds are applied to prevent anyone viewing a report or exploration from inferring the identity of individual users based on demographics, interests, or other signals present in the data.

My question is: is our reporting being concealed because GA thinks we're being too intrusive with the personalized answer tracking? Or is this something more benign, like an error on our implementation? We have 277k answers, so I'm not sure how we could infer individual identities from such a large sample-size. Plus our quiz never asks for any personal information, like name, location, age, nothing... just 4 answers and that's it.

What could be causing this thresholding?

Update: I've changed the Reporting Identity from By user ID and Device to By Device only, but the problem persists.

Answer 1

When you see this it typically means that sampling is being applied. Thresholding is only applied when you're not running default reports (not subject to it), and yours don't seem default.

When running ad hoc reports I've experienced thresholds in the following circumstances:

1. The sample is too small (you seem confident this not the case)

2. The sample is too large (277k events = how many sessions)?

   • With Analytics standard I believe that the sampling threshold on ad-hoc queries is 500,000 sessions for a given date range. Analytics 360 is much larger (millions)

3. The date range is too narrow

4. When Google Signals is set in the reporting identity options for a property

   • I nearly lost my marbles once trying to figure out what was going wrong before I considered this but it seems like you already have given your update to the post.

5. When compliant data has been falsely flagged. This I had to take up the issue with Google to figure this out (edit: see below re: contacting support).

If you've spent more than an hour going through Google's docs there's a good chance that none of the above are helpful. Unfortunately these issues are so dependent on the individual site/app/property.

I can't really guess whether it's benign or not, but it is a personality test, so it's hard to rule out that a machine might consider aspects of it to violate the terms. If you believe you might have been falsely flagged, you can get in touch with Google Analytics support via chat (that's what I did re: #5).

This might take forever without being a paying customer. I'm fairly certain the reason I had a slightly better than terrible experience is because I purchase advertising products (PPC for clients).

Support is available only to Sales Partners and to customers that purchase Google Marketing Platform advertising or Analytics 360 products directly from Google.

- Google

Answer 2

GA is a free tool. Conducting queries on large amounts of data is expensive. Therefore, thresholding and sampling. You make your request cheaper by typically applying smaller data ranges, so that could be a quick fix to the problem.

For a "proper" fix, you can consider a paid version of GA, but that makes no real sense for a simple quiz. Since you're a developer, you may find it easier to set up data export from GA to big query and then conduct your analysis using SQL.

A lot of advanced analytics teams who choose to stick to Google's stack of analytics (as opposed to paid Adobe stack or free Matomo stack) would do exactly that, but then ETL the data from BQ (since BQ will start charging you after certain query threshold) into their own data lakes like Snowflake or Azure, or whatever. After they have daily ETL from BQ to their custom data lake, they would typically use a BI tool like Looker or PowerBI on top of the data lake to analyze the data properly, building live dashboards, sharing them with stakeholders and whatnot. This, again, might be an overkill for what you're doing.

There are also marginal solutions to the problem. GA has a reporting API for anyone to pull the data. It's pretty uncomfortable to use, but it ultimately works. There are also a few scripts/services that allow custom data exports to certain destinations. I've seen and used extensions/plugins for Excel and Google Sheets, but there are certainly more options. There also should be open source packages to get the data like this python client from Google.