A/B testing is a core part of any successful developer's monetization playbook. To improve your KPIs, you should test just about everything - from rolling out new ad units and optimizing waterfalls, to deciding if in-app bidding is right for you. By A/B testing changes in your monetization strategy, you can make data-driven decisions, minimize risk, and increase revenue. Here are 5 of the most important A/B tests you should run.
1. Test in-app bidding
The benefits of in-app bidding for developers of all sizes - like higher revenue and reduced operational overheads - are clear. Even so, migrating your game, or portfolio of games, to a new monetization model is a big decision. That’s why it’s important to A/B test before fully switching to bidding, to understand if bidding is right for your game and remove any uncertainty about its impact on your monetization performance.
The key metric to look for when A/B testing bidding versus waterfall monetization is ARPDAU. You should also keep a close eye on metrics like eCPM and impressions. If these KPIs are performing stronger in the bidding test group, you can be confident that bidding is right for your game and begin to roll it out fully.
You’ll be able to find these KPIs in the reporting dashboard of your mediation platform, and even run the A/B test inside the platform, too. For example, ironSource’s bidding test tool helps you come to a fast decision about your monetization strategy, with minimal effort. Instead of spending time creating new waterfalls or new instances, you can simply turn on the bidders and click a button to start the A/B test. This means you can start increasing revenue as soon as possible, without wasting time on operational overheads.
Rose Agozzino, Senior Marketing Specialist at Ludia said: “After running A/B tests with the ironSource team to determine the impact of LevelPlay on our profits, we decided to combine bidding with our traditional waterfalls. This hybrid approach automated much of our operations while boosting eCPMs, in other words - saving us more time and increasing our revenue”.
"This hybrid approach automated much of our operations while boosting eCPMs, in other words - saving us more time and increasing our revenue”
Rose Agozzino, Senior Marketing Specialist at Ludia
Use ironSource’s quick bidding test to determine if bidding is right for your game
2. Test waterfall optimization
Bidding is quickly becoming the default for monetization, but the industry is still operating in a hybrid mode that combines bidding with traditional waterfalls. Making sure your waterfalls are fully optimized is crucial for maximizing revenue - and A/B testing is the best way to achieve this outcome.
There are numerous waterfall settings that you can A/B test, such as country, different instance pricing, adding or removing instances, and groups. Groups allow the flexibility to control your waterfall based on country and user segment - or a combination of both.
These settings can have a significant impact on your bottom line. Adding a new network, for example, could bring in an influx of highly relevant demand for your ad space; while changing your price points on the instances in your waterfall could help unlock higher scale and demand.
ironSource’s quick A/B testing tool is the easiest way to test every type of waterfall, and lets you test all of the settings we mentioned above. How does it work? The tech does the heavy lifting, automatically dividing traffic 50/50 into two groups, and uses data based on real revenue from the reporting APIs to ensure accurate results. Performance KPIs are totally transparent and can be broken down from day one, so you can track your results and end the test as soon as you’re convinced.
The quick A/B testing tool’s biggest pull is its ease of use: it simplifies what is often a complicated and time-consuming process, and lets you change anything you want from your mediation management page.
Use ironSource’s quick A/B testing tool for optimizing waterfalls
3. Introducing a new ad unit
In addition to testing mediation management, make sure to also test new ad units. If there’s an ad unit you’re not leveraging in your game, consider testing it to a limited audience - it could have a positive impact on your ARPDAU or retention rates - or sometimes both.
The strongest monetization strategies combine a mix of ad units, for example, user-initiated ad units like rewarded video and offerwall, and display ad units like interstitials and banners.
Different ad units appeal to different types of gamers, and users respond differently to, say, an offerwall than they would to a rewarded video. As Natalya Kopyrova, Ad Monetization Manager at ZiMAD said: “After switching to ironSource's mediation, there was no way we could have known how well interstitials would have performed on our game without the ironSource testing tool.”
"There was no way we could have known how well interstitials would have performed on our game without the ironSource testing tool”
Natalya Kopyrova, Ad Monetization Manager at ZiMAD
Use ironSource’s robust A/B testing tool for testing out new ad units.
4. Test capping and pacing
How often can you show players system-initiated ads in order to maximize ARPU, without causing any damage to retention? How many times in a session should you give users access to rewarded video ads, without hurting your in-game economy? The answers to these questions can be found through A/B testing pacing and capping. Pacing is the time in between each ad you show, and capping is how many ads you show a user in a single session of gameplay.
Look out for impressions, retention, and ARPU, and allow enough time and a large enough volume of users before stopping your tests - around 2 weeks is usually enough.
Test what happens when you increase the frequency of your system-initiated ads. If after two weeks you see that D1 retention dropped, without a big enough increase in your ARPU, then try reducing the frequency or changing up the placement. But if you increase the frequency and you see ARPU rise after a couple of weeks of testing - with no significant impact on retention - then you can roll out this change to your full audience. Remember to research the benchmarks for your specific genre to see how your metrics perform relative to the rest of your category.
Neon Play put this advice into practice and were able to increase ARPDAU without hurting retention. CEO Oli Christie said: “We never considered testing as standard practice, but with ironSource it’s so easy, so now we do it all the time for each game.” Read the full case study here.
“We never considered testing as standard practice, but with ironSource it’s so easy, so now we do it all the time for each game”
Oli Christie, CEO, Neon Play
Use ironSource’s robust A/B testing tool for testing capping and pacing
5. Experiment with ad content
A huge factor behind the success of your ad monetization is the content of the ads you’re serving your users. One question that developers commonly ask is if they should serve ads from other games in the same genre.
Jarkko Rajamaki, VP Advertising at Rovio, explains that users who install games from the same category actually have the highest retention rates. That’s why it’s highly recommended to A/B test how ads from similar games affect retention - a positive uplift to retention helps maximize the LTV of your users, which is a KPI of any monetization strategy.
Hopefully, you’ll find that showing ads from other developers in your space will have a positive impact on revenue, but won’t damage the retention of users. That’s because your users will often enjoy the ads, leading to more engagement, higher eCPMs, and more revenue.
You can also experiment with the timing of the X button on interstitial ads. For example, test the impact of the X button of an ad appearing after 3 or 5 seconds. A/B testing the timing will help find the sweet spot - ensuring that players have enough time to engage with the ad, but not too little time if they want to click away and get back to the game. Ask yourself: what will generate the most revenue but not frustrate users and damage retention?
Use ironSource’s robust A/B testing tool for testing new ad content
The power of data
Ultimately, A/B testing is a means to reduce uncertainty and make great business decisions based on data. There are so many parts to game growth, and knowing where, what and how to optimize can seem overwhelming - but A/B testing with tools like those offered by ironSource makes this process a whole lot easier.