Quick A/B test
What is a quick A/B test?
The quick A/B test enables you to test different waterfalls without any additional setup.
Traffic is automatically allocated 50/50 between the A and B groups. Performance KPIs can be broken down by A vs. B in the platform Performance, User Activity, and Bidding report pages to compare results.
What can you test with the quick A/B test?
You can create any kind of waterfall test with the quick a/b test. You can test different waterfall settings including country groups, new networks, different instance pricing, and cross promotion.
How the quick A/B test works
Once activated, the quick A/B test divides traffic equally between groups A and B. At this stage, the A and B groups are identical and no changes will be applied to either group.
- Each group will include all of your active instances and mediation groups
- The groups are identical in settings and waterfalls
- No pre-test setup is required
How to start a quick A/B test
Step 1: Navigate to the A/B Testing page
Click the Create Test button next to the Quick A/B Test option
Step 2: Name and activate the test
Give the test a name and description. Then click the Start Test button to start the test.
Step 3: Go to your Mediation Management page and modify each group’s waterfall settings
You’ll see a toggle on the mediation management page that allows you to switch between groups A and B. Adjust the waterfall settings for the test group according to the variables that interest you. Each change you make will apply ONLY to the group you’ve selected.
Supported waterfall components include:
- Capping
- Addition (or subtraction) of instances from the mediation group
- New group
- Delete group
- Duplicate group
- Export CSV
- Rate
Step 4: Terminate the test
Once you’ve analyzed your results and reached a conclusion about which group performs better, you can choose the winning group and terminate the test by clicking on the Continue with A or Continue with B buttons.
Terminating a test means the winning group’s settings will apply to 100% of the traffic on your app.
Note: Once you’ve terminated a test and approved the winning group, this action can’t be undone.
Monitoring and reporting
To monitor an active bidding test, use the platform reporting pages—Performance, User Activity, and Bidding reports.
Breakdown by A/B is available on the Performance, User Activity, and Bidding reports, with all metrics supported. To view reports broken by A/B, select one app that completed or is currently running an A/B test.
Click Show Reports to view the Performance report with the app pre-filtered, test start/end dates selected, and the data broken down by A/B.
Here’s an example of a Performance Report that’s broken down into A and B groups:
