How to Set Up A/B Testing in Marketo

A/B testing is a crucial aspect of email marketing. As the name suggests, A/B testing allows you to send two or more variations of an email to different subscribers. According to the performance of these campaigns, you can then send the winning version to the remaining subscribers. 

Every ESP or CRM has a different interface according to which the steps for A/B testing vary. Here, we shall discuss how you can carry out A/B testing through Marketo. We chose Marketo because it is one of the most robust platforms and is quite popular among the masses.

If you are looking for a step-by-step guide to A/B test your Marketo templates, this article will clear all your doubts about it.

Let’s get started.

First of all, we shall understand how to add an A/B test.

1. Adding an A/B test

Before you can add an A/B test, make sure you fulfil these preconditions. 

• Create an Email Program.
• Define an audience either by a Smart List or by importing a list.
• Choose an Existing Email or Create an Email for an Email Program. 

Once you have completed these steps, you are ready to execute A/B testing. 

Just follow these steps below:

Navigate to Marketing Activities. 

Choose your email program.

You will be able to see “Add A/B Test” under the title “Email”.

Click on that and Voila! You are good to go…

Now, we shall move on to the different test types. 

a. Subject line A/B testing

Once you click on “Add A/B Test”, a new window will open that will allow you to choose the test type. 

Click on the Test Type drop-down and select “Subject Line”.

The Choice A will show you the subject line of the selected email.

In the Choice B, you can write the subject line of your choice for the test email.

If you want to test more emails and add more subject lines, just click on the plus sign. 

Make use of the slider to select the percentage of the audience who must receive the A/B test. 

Then, you will define the A/B test winner criteria. (We shall discuss it later in this article.)

b. Whole Email A/B Testing

Just like you have selected ‘subject line’ in the previous point, select the “Whole Emails” test type to test the entire emails.

Just select the emails that you want to include in the A/B testing.

Move the slider to adjust the test sample size and click on Next.

c. From Address A/B Testing

Let’s say you want to determine whether “Email Uplers” works better as the from name or “Kevin from Email Uplers”.

From Address A/B testing comes into picture here. 

Choose “From Address” for Test Type and then select the From names and email addresses. 

For example: Kevin from Email Uplers, kevin@uplers.com

OR

Email Uplers, email@uplers.com

You can add more versions too, but avoid adding too many of them. 

Adjust the slider to choose the sample size and click on Next. 

d. Date/Time A/B Testing

Figuring out the best time to send the emails takes trial and error. You can take help of A/B testing and see what time works the best to drive email conversions for your business. 

All steps remain the same as in the previous points. Just choose “Date/Time” in the test type and pick the date and time when you want to send the test emails. 

Adjust the test sample size with the help of the slider and it is done. 

Now, once you have set the A/B test, you have to provide the A/B test winner criteria.

A/B Test Winner Criteria

Defining the A/B test winner criteria is the most important step in this process. 

Default winner criteria options are:-

• Opens
• Clicks
• Click to Open %
• Engagement Score

There is one more option known as “Custom Conversion” which will let you pick an event as a conversion with the help of triggers and filters. 

Just look for the trigger that matches your preference and drop it into the canvas. 

Next, you must define the trigger. 

Remember that you do not need to set the filter to “Was Sent Email”.

The triggers will apply only to the people who have received the email. 

Click on Close. 

Besides the winner criteria, you must also choose how you want to Declare Winner. 

It can either be Automatic or Manual. 

In the former option, Marketo can automatically send out the winning email at the scheduled time. On the other hand, if you choose the Manual option, you can review the results and then decide to send the email. The platform will wait for you to choose the winner once they send you the report of the results. You will have to define the date and time to send the report in case you have chosen the Manual option.  

Schedule the A/B Test

After setting up all the parameters, you must schedule the A/B test.

Pick the date and time to send the test email and winner email. 

Send Test and Send Winner must have a gap of 4 hours at least so that you get enough time to get the results. If you are sending the emails to a larger audience, you can wait for a day or so. 

You can even add notification recipients to get updates about the test results. 

The last step is to approve the program, provided that you have already defined your audience and chosen the email. 

To approve an email program, just go to “My Email Program” from the Marketing Activities and click on Approve Program under the Approval title. 

This is important because the email will not be sent unless you approve the email program. 

Points to Remember

1. You will not be able to use the test email for any other program. 

2. In case your database has duplicate records, those subscribers will get the test emails as well as the winning versions. Therefore, you must always find and merge duplicate people in your database. 

3. Do not set the sample size to 100%. In case you do so while working with a static list, the test email will be sent to everyone in the audience. As a result, there will be no winning version to send anyone. 

If you set the sample size to 100% while working with a smart list, it will send the test email to everyone in the database at that point of time.

Whenever the email program will run in the future, all the subscribers who will qualify for the smart list will receive the email as they are included in the database.

4. It is possible to add multiple emails but it is advisable to avoid this as it can slow the testing process. 

Conclusion

Email marketers who are struggling to figure out what is working for their business should certainly take the A/B testing route. It will help them to optimize their email campaigns for best results and enhance the conversion rate.

If you still have any questions about how to do it, just reach out to our Marketo experts and we would love to help you out.