A Beginners Guide to A/B Testing with Email
The people on your email list have already expressed an interest in your content and they may have even purchased from you already. Because it is much easier and less expensive to retain a customer than earn a new one, it is important to nurture your existing customer base through email.
In order to determine the best formatting or techniques for your email campaigns, you will need to use a more scientific process than simple guesswork. You may think your emails sent on Tuesday will perform better than ones sent on Monday, but have you tested that theory? The same idea applies to a variety of factors concerning your email campaigns, including calls to action, subject lines, and personalization.
What is A/B testing?
A/B testing is a concept frequently used by marketers to compare the effectiveness of two things side by side. Also known as split testing, this process involves running an experiment and comparing two versions of something in order to decide which option performs best. For webpages and emails, A/B testing involves comparing a variation against the current experience, collecting data, and using that information to change your strategy.
Why is A/B testing important?
It is easy for marketers to assume why a particular result was achieved or make assumptions about their audience’s behavior. This can be especially true for marketers that have been in the game for a longer period of time or professionals who feel like they know their target audience inside and out. We form opinions and guesses all day every day, but that does not mean our hypotheses are always correct.
A/B testing allows you to prove your hypotheses as right or wrong. It does not leave marketers to rely on intuition or outdated information in order to make decisions on effective marketing campaigns. Instead, A/B testing offers a way for marketers to improve the user experience and conversion rate on a consistent basis.
What can you A/B test with emails?
There are a lot of factors that go into your email campaign. Chances are slim that you are consistently thinking about each and every piece of the puzzle for your emails. You may spend time crafting the perfect subject line only to push send without paying attention to the day of the week or time of day. You may also find yourself incorporating high-quality images only to rush through the body copy.
Whatever the case, marketers can fall into the trap of autopilot or obliviousness when generating and sending an email marketing campaign. This is where A/B testing really comes in handy. A/B testing makes each decision intentional and continually compares new or updated versions against the current version to decide which performs better on a consistent basis.
When deciding what to A/B test, you are left with considerable options. Each element of your email or email campaign will have a different impact on your business or conversions. It is important to carefully consider what you will be testing before jumping in.
Consider testing the following elements for your email:
- Subject line
- Personalization
- Body text
- Images
- The message layout and placement
- Call to action
- Testimonials
- Headlines
- Offers or promotions
Who is your A/B test audience?
It is typically recommended to run A/B tests with your entire email list. This is the best way to get an accurate idea of your audience’s response to your emails or campaigns. If your A/B test is charged by email address, you may need to cut down your list to fit your budget. Also, if you are trying something very out of the norm, you may want to limit your test group in case the experiment does not go as planned. Small batches can also be beneficial for limited time offers.
Larger samples provide more accurate results and statistical significance. Split testing must be done at random in order to gain accurate results. Choosing which contacts on your list get a specific email can skew your results and invalidate your A/B test.
How do you define success?
It is important to have a goal for every A/B test before you start sending emails. Once you know what you are testing for and the audience you are using, decide on the definition of success. Your previous data and results can provide a baseline for your tests as well as average numbers to reference.
If you are trying to increase your conversion rate, consider the average conversion rate for your email list and decide on your new goal. This can also be applied to open rates or click rates. Whatever you want to accomplish, have a specific goal in mind.
Tips for A/B testing your emails
- Run your tests at the same time to eliminate time as an additional factor
- Use as large of a sample as possible to provide accurate results
- Go off of the data, not what you think is best
- Test often to continue improving your user experience and message
- Limit your tests to just one variable at a time so you do not muddy up your results
Tools for A/B testing emails
Your email software likely already has A/B testing capabilities built-in. This is true for Campaign Monitor and MailChimp. If you do not have this feature, you can set up manual tests by splitting your audience in half and send version A of your email to one half and version B to the other.
Conclusion
A/B testing is a powerful tool for marketers. Instead of playing the guessing game or making assumptions, you can put your hypotheses to the test and use empirical evidence to prove or disprove your theories. When you conduct these tests, you can use the results to refine your emails and email marketing campaign. Marketers should subscribe to the “always be testing” philosophy and continually running new A/B tests to improve their conversion rates.
