A/B testing & Experiments

In this article, you will learn how to:

• set up an A/B test;
• configure A/B testing settings and criteria;
• manage and monitor your experiments;
• filter and analyze experiment results.

How to set up an A/B test

Go to Campaigns > click on the Add button (top right corner) to add a new campaign > choose a channel and a respective template > Add A/B test

Add A/B variant

To add 2 more variants (max total of 4), click on the plus icon next to the variants.

Add another A/B variant

From this dashboard, you have options to edit the variant (the pencil icon), go back to Template library (the reload icon next to it), and to delete the variant.

Edit A/B variant

Clicking on the Edit icon will take you to HoodEngage Studio, where you can edit the element you want to A/B test.

Once you’ve decided to add a variant and initiate A/B testing, you’ll see a detailed explanation of how the A/B testing process works in the campaign preview panel on the right side of your screen.

Here, you can also change your A/B testing name to be able to find it later in Experiments.

How A/B testing works in HoodEngage

• Each of your 2 variants will be tested with 30% of your selected test group;
• The system will run testing and measure the criteria you set (click rate or open rate) for exactly the time you’ve set in Settings;
• When the testing finishes, the best-performing message will be sent either automatically or will wait for your decision which variant to send (depending on the A/B test settings) to the remaining 70% of your recipients.

A/B testing settings

To access A/B test settings, click on the cog icon in the top right corner of the A/B testing section.

A/B testing settings

Here, you can set up your A/B testing criteria depending on the channel you want to test:

• For push notifications is only available click rate;
• For email are available click rate and open rate;
• For SMS is only available click rate.

Then, you can choose whether you want A/B test to proceed automatically or manually:

• If you choose automatically, the system will send the winner to the remaining 70% of the audience without asking you for additional permissions;
• If you choose manually, the system will determine the winner but wait for your decision on which variant to send to the remaining 70% of the audience.

A/B testing options

After you launched your campaign, you will find everything A/B testing-related in the Experiments tab.

Experiments

The Experiments tab within HoodEngage represents a centralized hub for A/B testing.

To access Experiments, go to Insights > Experiments.

Here, you will find an overview of all your A/B tests, together with its status, name, type, variant, start and end dates, and result.

Experiment overview

Also, you have the option to easily find your A/B test by filtering it by its end time, type (SMS, Push, or Email), and Result. Or, use quick filters to filter them by status.

Filter experiments

By clicking on a specific A/B test, you will get a detailed preview of your A/B test and its settings.

A/B test preview