Did it actually work? How causal impact analysis can measure marketing investment

In summary

• What: Causal impact analysis helps estimate the effect of a specific marketing, measurement or technology change by comparing what happened with what likely would have happened without it.
• Why: Performance can improve after an intervention without that intervention necessarily being the cause. Causal impact analysis provides a more rigorous way to assess the difference an investment actually made.
• Where it can help: Louder sees applications across server-side tagging, CAPI, new marketing channels and strategies, bidding changes, consent and CMP implementations, website improvements and other measurable interventions.

When better numbers don’t tell the whole story

Marketers make changes to improve performance all the time, from switching bidding strategies and introducing new channels to implementing server-side tagging.

Then the numbers move. But how do you know the change actually caused it?

There are plenty of other factors at play. Demand may already be growing. A promotion could drive more people to the site. Media activity may have shifted. Or better measurement could simply be capturing activity that previously went unrecorded.

This is where causal impact analysis becomes useful. It estimates what would likely have happened without the intervention, creating a counterfactual that can be compared with what actually happened.

In other words, it helps move the conversation from “performance improved after we did this” to “what difference did this actually make?”

We’ve already seen this in practice at Louder. For one automotive client, our analysis estimated that implementing server-side tagging improved GA user measurement coverage by 15.2%.

More on that below.

What is causal impact analysis?

Causal impact analysis uses historical patterns and suitable comparison data to model a counterfactual: an estimate of what would likely have happened without the intervention.

Comparing that estimate with what actually happened helps quantify the likely effect of the change, while accounting for uncertainty around the result.

Imagine a business records 1,200 leads after changing its media strategy. The model estimates that, without the change, it would probably have recorded around 1,000 leads over the same period.

That suggests an estimated uplift of 200 leads, or 20%.

It doesn’t mean we can definitively say exactly 200 leads were caused by the intervention. The result is an estimate, with a range of plausible outcomes and assumptions that need to be considered when interpreting it.

The overarching approach uses time-series modelling to construct the counterfactual and estimate the effect of an intervention over time.

Where can causal impact analysis be used?

This is where we think the opportunity gets particularly interesting.

Causal impact analysis doesn’t have to be limited to assessing campaigns. It can help evaluate changes across marketing, media, measurement and technology.

Some potential applications include:

Change	Question causal impact can help answer
Server-side tagging	Did sGTM improve measurement coverage and recover previously unmeasured activity?
Conversions API	Did CAPI improve conversion capture or the quality of signals being sent to advertising platforms?
New media or bidding strategy	Did the new approach materially improve performance?
Consent banner or CMP	What effect did the implementation have on consent, measurement coverage or observable behaviour?
Website improvement	Did improving site performance increase customer completion?
Checkout or form redesign	Did removing friction actually improve conversion?
CRM audience activation	Did activating first-party audiences generate additional customer response?
SEO or content changes	Did the change materially improve organic performance or downstream outcomes?

How Louder used it to measure server-side tagging

Louder recently used causal impact analysis to evaluate a server-side Google Tag Manager implementation for an automotive client.

The question wasn’t simply whether website traffic increased after implementation.

We wanted to understand: did server-side tagging improve the amount of existing user activity being captured in Google Analytics?

The analysis used daily data from July 2025 to May 2026, with server-side tagging deployed in February 2026.

Total users in GA was used as the primary outcome, while a range of other factors were considered, including organic search activity, paid media across SA360, DV360 and CM360, CRM leads and sales, and normal day-of-week patterns.

We checked for correlation and multicollinearity, and tested multiple model specifications rather than relying on a single combination of controls.

The preferred model compared actual GA users following implementation with an estimated counterfactual representing what measurement would likely have looked like without server-side tagging.

What did we find?

The analysis estimated that server-side tagging improved GA user measurement coverage by approximately 15.2% during the post-implementation period.

That equated to around 512,000 additional measured users, with the model estimating a plausible uplift range of approximately 11.6% to 19.1%.

Importantly, server-side tagging didn’t create 512,000 new visitors.

It meant a significant layer of existing activity was now more visible in GA.

For the client, that meant greater confidence in reporting and a more reliable measurement foundation for assessing campaign performance, media efficiency and downstream business outcomes.

Putting a number on the investment

One of the biggest opportunities for causal impact analysis is putting a measurable outcome around investments that can otherwise be difficult to quantify.

A server-side tagging implementation, CAPI rollout or CMP change has a clear implementation cost. Demonstrating what the business received in return can be much harder.

Causal impact analysis helps move the conversation beyond “the numbers went up afterwards” towards “here’s our statistical estimate of the difference this intervention made.”

It can also help businesses avoid giving credit to the wrong thing.

A campaign launched just before a seasonal peak may look successful simply because demand was already increasing. Conversely, a useful intervention could be overlooked because the wider market declined at the same time.

And an inconclusive result can be useful too. It might tell you the effect was smaller than expected, the data was too noisy or the evaluation needs a stronger design.

All of those findings can help inform what happens next.

What do you need to run causal impact analysis?

The strongest analysis starts before the intervention happens.

Ideally, teams should establish:

• A defined outcome: What does success actually mean? This might be qualified leads, backend sales, measured users, revenue or completed bookings.
• Data before the change: Enough history to understand normal patterns and determine whether a credible model can be built.
• A clear intervention timeline: When did the change happen, who or what was affected and was it rolled out all at once or progressively?
• Data after the change: Enough time to observe what happened once the intervention was in place.
• Suitable comparisons: Unaffected markets, brands, page groups or external demand indicators that can help estimate what would have happened otherwise.
• Business context: Promotions, pricing, stock availability, media activity, holidays and other factors that may have influenced performance.
• Consistent measurement: Confidence that apparent changes can be meaningfully interpreted.

There isn’t a universal minimum number of weeks or months.

The amount of history required depends on factors including seasonality, noise, expected effect size and the quality of comparison data available. More data also doesn’t necessarily compensate for poor controls.

That is why feasibility should be considered before deciding causal impact analysis is the right methodology.

So what about MMM and MTA?

Causal impact analysis isn’t a replacement for marketing mix modelling (MMM), attribution or experimentation.

They answer different questions.

Approach	Main question	Typical inputs	Practical role
Causal impact analysis	What difference did this particular change make?	Outcomes over time, rollout dates and credible comparisons	Evaluate a rollout, campaign decision or technical change.
Marketing mix modelling (MMM)	How do marketing and other factors contribute to outcomes, and how should investment change?	Aggregate outcomes, channel activity or spend, and relevant business drivers	Support broader budget allocation and planning.
Multi-touch attribution (MTA)	How should conversion credit be distributed across observed customer touchpoints?	Linked interactions and conversion paths	Help understand journeys and inform tactical decisions within the coverage of the data.

MMM itself can also be designed for causal inference. Google’s Meridian, for example, uses control variables and geographic data as part of its modelling approach.

There is no single model that answers every marketing question.

Sometimes a planned experiment will provide the strongest evidence. Sometimes MMM is the better approach. And sometimes an organisation has made a defined change and wants to understand the impact that particular intervention had.

The starting point should always be the business question you’re trying to answer and the evidence available to answer it.

Louder’s recommendations

Plan the measurement at the same time as the investment.

If you’re considering server-side tagging, CAPI, a new marketing channel, a CMP implementation or another significant change, don’t wait until afterwards to ask whether it worked.

Before implementation:

• Define the business or measurement outcome you expect to change.
• Establish the baseline and data you may need to construct a counterfactual.
• Identify other factors that could influence the result.
• Record exactly when and where the intervention occurs.
• Consider whether a phased rollout, holdout or other experimental design could provide stronger evidence.
• Decide upfront how the result will inform the next business decision.

Get in touch

Get in touch with Louder to discuss how privacy, consent, measurement and data governance obligations may impact your organisation.