Consented or modelled data - what to trust in GA4

I recently compared GA4's interface figures against the raw BigQuery export to answer a simple question a client's leadership had asked - can we trust this data? The answer is yes, but the reasoning is more useful than the answer, and it hinges on a key and non-obvious distinction.

Maybe you can relate to this - the other day I pulled a weekly report from Google Analytics' interface and it showed Organic Search traffic was down nearly 50% versus the previous week. Nothing had changed on the site, and nothing changed with the tracking. Someone spots this and then blames the tracking, or says they don't trust the GA data. I'm sure I'm not the only one who deals with this ALL THE TIME.

Nine times out of ten this is due to the fact that the GA property in question uses the "Blended" Reporting Identity, which mixes observed (i.e. consented) data with Google's behavioural modelling (i.e. estimates). The outcome is that the reported figures are the 'best guess' based on a black-box machine learning algorithm Google doesn't like to share details on. It's more of a "trust us bro" mentality Google takes, for better or worse.

But explaining that to a senior leadership team isn't easy, nor is it practical. What they hear is "some of the data is made up", and from that point every number from GA gets treated with suspicion - including the ones that are counted correctly. And so starts the constant back-and-forth about whether or not the GA data can be trusted. So what started as explaining the observed change in reported numbers, ended up being a deep-dive into establishing (with evidence) which figures in GA are exact and which are estimated, and thus, what they can trust.

'Trust' being the operative word here. I don't like using the term 'accuracy' in marketing these days as it's no longer relevant. Questions like "is GA accurate?" literally cannot be answered (and is likely 'no' to be honest), but we have no choice and there is rarely a 'more accurate' source for marketing analytics - they are all pretty much the same. But questions like "can we trust GA?" are important, and this is what we can (and should) address whenever it gets asked.

Part 1: Consent Mode and behavioural modelling

The whole concept of 'advanced' Consent Mode (CoMo) is that when someone declines analytics cookies on a website, GA still tracks the events. No cookie(s) are set without consent, so GA can't tie these 'cookieless pings' to a user or a session. These unconsented events are useful as GA's behavioural modelling uses them as training data for its ML model to estimate the users and sessions of the unconsented users. And when using the "Blended" Reporting Identity, that estimated data is included in all of the reports.

The other very useful part of advanced CoMo is that all of the events (consented and unconsented) are exported to BigQuery in the daily tables. This means that we can see exactly how many pages were loaded and purchases were made by looking at the total count of their corresponding events page_view and purchase. This goes for any event except for the session- and user-based ones session_start, first_visit and first_open.

So we have a way of seeing the modelled reported events via the interface (or API), and we have a way to validate them via the BigQuery tables. Let's test it...

Part 2: Pulling the data

The GA BigQuery export is the reference point for the reason mentioned above - it contains all events and no modelled data. And for each event in the table(s), we can see the consent state recorded in privacy_info.analytics_storage.

We start by querying the BigQuery events table, pulling the events you want to validate, then export or copy to a Google Sheet. Below is an example SQL query to use, where you can add any events you want to look at:

SELECT
  PARSE_DATE('%Y%m%d', event_date)                            AS date,
  COUNTIF(event_name = 'page_view')                           AS page_view_total,
  COUNTIF(event_name = 'page_view'
          AND privacy_info.analytics_storage = 'Yes')         AS page_view_consented,
  COUNTIF(event_name = 'purchase')                            AS purchase_total,
  COUNTIF(event_name = 'purchase'
          AND privacy_info.analytics_storage = 'Yes')         AS purchase_consented
FROM `project.analytics_XXXXXXXXX.events_*`
WHERE _TABLE_SUFFIX BETWEEN '20260801' AND '20260831'
GROUP BY date

Note: Use event_date, not event_timestamp. event_date is in the property's reporting timezone, which matches the interface and API 'Date' dimension. event_timestamp is a UTC timestamp, and can cause issues when the reporting time zone is not UTC.

Pulling the interface/API numbers is the easy part - use the events report adding Date as a secondary dimension, or use a free-form exploration. Then export the data to a Google Sheet, and copy over the sheet with the BigQuery data. It should go without saying that you need to be using the 'Blended' Reporting Identity, and modelling needs to be active.

I'm not going to go into the nuance of VLOOKUP and SUMIFS in sheets, you can figure that out. But the gist is that we want to have both the BigQuery and interface GA data side by side in a table, so we can visualise it and spot trends.

Part 3: Validating the event data

I like to visualise the data in this way for each event:

As you can see from the data showed above, the event counts in the interface are not modelled, even when using the blended reporting identity. They match the BigQuery total counts almost exactly every day.

As stated previously, this does assume the property is eligible for modelling. If yours doesn't clear GA's modelling thresholds (approximately 1,000 consented and unconsented users per day), then the blended reporting identity has nothing to blend. So even with it selected, you see consented event counts only, and they reconcile to the consented slice of the export rather than the full total. Annoying I know, the data is there, it's just that GA does not surface it in the interface/API reports for some unknown reason!

It's also interesting to check that the consent rates are stable. The more stable they are, the more confident the modelled data can be, as it cannot predict fluctuation if you have 'noisy' data, or unpredictable seasonality periods. This is not a problem for the property in my example, as they have ~60% consent rate for the page_view across the whole period.

So the tracking was doing its job the whole time. What people did on the site, what pages they saw, and what they bought, was counted correctly and reported correctly.

Part 4: Mind the (session) gap

tl;dr - Don't ever use the session_start event.

Fine, the longer version. Now let's talk sessions... You might be tempted to do the same process as above for the session_start event, but that would not be wise. Technically every page load triggers a session_start event where the user denied analytics cookies. The way GA knows if a session has started is based on the time difference from the user's last logged event. But the only way GA knows when the user's last logged event was, is by the cookie, which doesn't exist due to denied consent!

But this is not the problem I have with the session_start events... My issue is that we get two ways to quantify sessions - counting the session_start event, and a distinct count of session IDs (technically a HLL++ approximation). And the interface/API gives both. But the event count is susceptible to the issue mentioned above, in that we are way overcounting the events as one is logged on every page load with denied analytics consent. That's not the case for the distinct session ID count - that one is modelled.

Just to prove this point, see the data below where we have the same session_start total event count in BigQuery and the interface, but that is different to the 'Sessions' metric in the interface:

So yeah, don't ever use the session_start event in the interface/API report if you use blended data, and never use it in BigQuery either. Just don't.

It's not all about sessions though, users also suffer from the same problem. In GA, a user is a cookie (a simplified explanation I know) - so no consent, no cookie, no user. The multi-touch attribution data is also in the same boat - no consent, no sessions, no user journey to attribute across. In short, all user- and session-based data is estimated, including attribution.

That's where the Organic Search drop lived in the original request that I got asked. The number of things that happened didn't change, but Google's estimate of how many people did them, and which channel they arrived from, had. Nobody outside Google can say why, because the model is a black-box.

Part 5: Trust is not the same as accuracy

This is the distinction the exercise clarified for me.

For the counted numbers (i.e. events) - accuracy is provable and trust follows from it. That's the easy case.

For the modelled user and session numbers, accuracy can't be established. There's no source of truth to check an estimate of unconsented sessions against. But that doesn't make the numbers untrustworthy - just a different kind of number. Google's model is applied consistently, it's the same approach almost every marketing platform's reporting now relies on, and over any sensible period it's directionally sound.

"Is this accurate?" has stopped being the useful question in marketing measurement. "Can we trust it?" is the one that matters.

Consent, browser restrictions and modelling mean a good proportion of what any platform reports is now an estimate. Insisting on accuracy across the board means either distrusting everything or pretending the estimates are counts. Neither is helpful. The workable position is to know which of your numbers are counted and which are modelled, hold the counted ones to a higher standard, and use the modelled ones for trends and comparisons rather than for precision.

Part 6: In reality

In reality, most of the data we use in marketing analytics is session- or user-based. So that means most of the data we use will be modelled and unvalidatable. But we can build trust in the data by validating the ones and zeros (events). Accuracy is a redundant term in marketing, but that doesn't mean we shouldn't be able to build trust to make decisions. And decisions are by far the most important thing, as without that, there's no point collecting the data to begin with.

For the marketer who asked me that question about the change in Organic Search traffic, the outcome was a short email saying:

The counts of what people did are exact, but the counts of who did it and where they come from are Google's estimations. And although these can move without anything on our side changing, they are stable enough to use for direction.

The daily comparison is cheap enough to keep running in the analytics engineering stack, and it's becoming the first table I build in the reporting layer so the evidence is always current rather than a one-off.

Now, the logical process and explanation is easy enough, but the hard part is the education and hand-holding for the wider business that actively use this data. All I can say on this front is it's a slow process and it needs repeating many times, but is worth it. And if it's not something they understand, just switch to the 'Observed' or 'Device-based' reporting identity and be done with it. I'm a fan of the modelled data, but only if it answers more questions than it raises. If it's raising more, it can sow the seeds of distrust, which is just not worth it.