Host Jess Overton, Director of Demand at ironSource Aura, is joined by Maor Sadra. Maor is a veteran of the ad tech industry with a wealth of experience - the most recent of which is founding the incrementality measurement platform, INCRMNTAL. Jess and Maor had a wide ranging discussion about mobile app marketing, incrementality measurement - including tips for getting started - and predictions for the future.
Listen to the podcast or read the edited highlights below.
Old school incrementality testing vs how INCRMNTAL do things
"Old school incrementality testing will usually tell advertisers, "Hey, stop advertising for a week. And we're going to tell you if your advertising actually created value or not".
And if you look at some of the uber famous cases of fraud in the industry, for example, the Uber case - Uber paused advertising and then saw that their numbers were not going down. Hence their advertising was basically cannibalizing their organic customers. You have a lot of other stories like this recently, like Airbnb, which found that $650 million of their marketing spend was literally completely redundant.
Now what we're doing at INCRMNTAL is different. So our approach is we don't want the advertiser to stop advertising. We think it's unrealistic to ask an advertiser to stop advertising. Plus there's all this potential bias when you do blackout tests and suddenly there is, you know, a global event or an external event or a competitor, and it affects your tests and you don't really know why things actually happen.
So we found an approach called causal inference. It's an algorithmic approach that allows us to understand what should happen if a customer wouldn't actually do certain change and versus what did happen. This allows us to do continuous measurement. So we're basically able to continuously show the marketer - this channel, this campaign, this activity created value, or didn't create value, or is taking value from somewhere else."
The rise of incrementality testing as a hot topic
"I think fraud probably was one of the driving forces. Nothing against the attribution companies, but in a way when attribution gives credit based on traffic, it kind of loses the context and essentially.
You're incentivizing certain bad players to get attributed to get the credits. I've seen so many cases where advertisers basically ran with a media provider. Everything looked good on an ROI or on a ROAS standpoint, but when the advertiser paused the campaign naturally - maybe budget ran out or whatever - advertisers started noticing that even though they have top notch attribution solutions, they're still getting hit with fraud all the time.
In addition, when Apple made their change, it damaged how attribution works - simply because you're now unable to do this on a single user level, and you're getting it on an aggregated level instead. This means your lowers your confidence level in deterministic attribution, so you're desperately looking for alternative solutions. All of these factors made incrementality more popular."
Common misconceptions about incrementality measurement
"I think that the most common concept people thought when they saw incrementality was retargeting. And I would say for retargeting, it makes sense to actually use incrementality using randomized control groups. Incrementality testing tools are offered by media vendors like Facebook and Google - the challenge there is the bias.
Even if you create this controlled test group, you have absolutely no control to see whether or not their users were affected by any other mediums. Now, basically to do it with an audience split on the impression and the click level makes sense, especially when you want to test, let's say, the effectiveness of a creative, or what creative is generating more incremental engagement on the ad level by users. And this, you could probably also use with aggregate level data.
But true incremental measurement is not about the traffic - it's about the value.
Now, a couple of other misconceptions. So incrementality measurement does not replace attribution. It cannot - it relies on attribution. It also doesn't replace media mix modelling. Media mix modelling is a more old-fashioned offline method to understand on a macro level, what affects your marketing, and this can be political, economical, your pricing versus your competitors, etc. For media mix modelling you usually need like a year long of data to do an analysis. I would say where we are is somewhere in between.
These methods aren't better or worse than each other - they are complementary. If a customer is doing offline and online marketing, they should use real-time attribution, media mix modelling, and incrementality testing."
Tips for starting incrementality testing
"Scale matters. An advertiser spending 20K a month only on Facebook likely doesn't need to do incremental testing with anyone. They could also probably afford to pause advertising for a couple of days, reactivate it constantly, and challenge their own spend to see if it's really creating incremental value.
Now the higher the scale, the more likely you need the help of technology. Fast growing companies rely on their marketing spend and it becomes kind of like a continuous effort to acquire new customer and to engage existing customers. The current tools in the market just show you direct performance. Now direct performance is confusing. Take ROAS, return on advertising spend. You could have a campaign. Where your return on advertising spend is positive, but if you would pause this campaign, if your results do not decrease, then in reality, while your ROAS was positive, your ROI was negative.
Advertisers or developers who are advertising at scale are already looking at how to do measure incrementality. Usually the challenge is coming up with the right time series data, which requires a lot of data science research."
Scientists vs. marketers
"If you compare scientists versus marketers, scientists will not necessarily believe what their eyes tell them because they're going to believe the science and the facts.
And I think this is how marketing is evolving. So on the one hand, marketers need get more creative and artistic to pull users to their product with messaging, and on the other hand they need to understand that often what they see in reports doesn't necessarily represent the entire truth.
Now I don't believe that marketers will go away, even in a world of automation. I believe automation helps marketers do what they're supposed to be doing. But it's the humans who will still need to be the ones setting strategies and making the decisions. But there will be a greater emphasis on a more scientific approach to understand how things work in order to actually get to the goal they want - and test, test, test.
One of the things that might go away with iOS 14.5 and ATT is you can probably no longer just put a budget on Facebook and it works because Facebook has everybody's data. I think this changes, and it's actually an advantage to marketers because it forces them to learn what works, and what doesn't. When you're completely relying on a monopoly to basically tell you what works and what doesn't, you end up losing a lot more control than you think."
The next layer of innovation and trends
"We'll see more automation tools in general to help marketers do what they're supposed to be doing and eliminate most of the hassle. We'll also like see more marketing tech stack, data science, prediction, and so on going in- house.
And then on the media side, whether you call it demand or supply side, we'll see more all-in-one platforms.
I think everybody's kind of copying the Google approach: having the entire ad stack in your advantage. It lowers costs significantly, and it basically allows marketers to get better performance and more trustworthy platforms they're working with. From a vendor standpoint, you can actually utilize first party data when you have owned and operated media and tools.
So it's just gaining efficiency. We've seen it recently with the rush towards M&A, with many companies eating other companies or merging with other companies. I think this is a sign of the growth or maturity of the market. That's one of the trends I think we will be seeing more of."
