Quick A/B test
What is a quick A/B test?
The quick A/B test enables you to test different ad monetization strategies without any additional setup.
Traffic is automatically allocated 50/50 between the A and B groups. Performance KPIs can be broken down by A vs. B in the platform Performance, User Activity, and Bidding report pages to compare results.
What can you test with the quick A/B test?
Waterfall A/B tests
Test different waterfall settings including:
- Country groups
- New networks
- Instance pricing
- Cross promotion
Ad strategy
Create any type of ad strategy test for all of the supported features on the ad unit and placement level:
- Ad unit capping and pacing
- Placement capping and pacing
- Rewarding
- Banner refresh rate
How the quick A/B test works
Once activated, the quick A/B test divides traffic equally between groups A and B. At this stage, the A and B groups are identical and no changes will be applied to either group.
- Each group will include all of your active instances and mediation groups
- The groups are identical in settings and waterfalls
- No pre-test setup is required
How to start a quick A/B test
Step 1: Navigate to the A/B Testing page
Click the Create Test button next to the Quick A/B Test option
Step 2: Name and activate the test
Give the test a name and description. Then choose the type of test you want to run: a waterfall setup test or an ad strategy test that features changes to ad unit capping and pacing. When you’re ready, click the Start Test button to start the test.
Step 3: Modify your test group’s settings
To test your ad strategy:
Go to the ad unit & placement page and modify the test group’s settings
You’ll see a toggle on the ad unit & placement page page that allows you to switch between groups A and B. To test your capping and pacing settings by ad unit and placement, adjust the settings for the test group according to the variables that interest you.
Each change you make will apply ONLY to the group you’ve selected with the following exceptions that will update in both A and B groups:
- ironSource instances
- Network test mode status
- Activation or deactivation of ironSource as a bidder
- Callbacks
To test your waterfall setup:
Go to your Mediation Management page and modify each group’s waterfall settings
You’ll see a toggle on the mediation management page that allows you to switch between groups A and B. Adjust the waterfall settings for the test group according to the variables that interest you. Each change you make will apply ONLY to the group you’ve selected.
Supported waterfall components include:
- Capping
- Addition (or subtraction) of instances from the mediation group
- New group
- Delete group
- Duplicate group
- Export CSV
- Rate
Step 4: Terminate the test
Once you’ve analyzed your results and reached a conclusion about which group performs better, you can choose the winning group and terminate the test by clicking on the Continue with A or Continue with B buttons.
Terminating a test means the winning group’s settings will apply to 100% of the traffic on your app.
Note: Once you’ve terminated a test and approved the winning group, this action can’t be undone.
Monitoring and reporting
To monitor an active bidding test, use the platform reporting pages—Performance, User Activity, and Bidding reports.
Breakdown by A/B is available on the Performance, User Activity, and Bidding reports, with all metrics supported. To view reports broken by A/B, select one app that completed or is currently running an A/B test.
Click Show Reports to view the Performance report with the app pre-filtered, test start/end dates selected, and the data broken down by A/B.
Here’s an example of a Performance Report that’s broken down into A and B groups: