How to A/B Test Your Direct Mailer, Step by Step

A/B testing is a randomized experiment to test the efficacy of two variants, A and B. A/B testing is used frequently in marketing to determine which of two pieces of marketing collateral performs better among target audiences. No matter what industry you’re in, if you do any sort of marketing, it’s a good idea to A/B test, particularly when you’re:

• Rolling out a new campaign
• Wondering why an existing campaign has stopped performing
• Curious about what will resonate with a specific audience, like a certain age bracket

A/B testing can be used for webpages, social media posts, advertisements, and so on. Today, we are going to show you how to implement an A/B test for a direct mail piece.

Set a measurable goal.

Before beginning any A/B test, ask yourself: How will you track success? Is it visits to your website through a unique URL included on your mailer? Is it phone calls? Sales? You won’t be able to track the success of either test variant unless you determine your goals up front.

Establish the control piece.

If you’ve sent out marketing postcards before and are just looking to refine, pick a mailer that has done well in the past. Show it to someone you trust and ask them what they would change about it. Does the call-to-action stand out? Should the image be switched out, or are more images needed? Is the color scheme attractive? Could there be less text on the page to highlight key points? Take this feedback into consideration and adjust your control piece accordingly. You don’t want your control to have obvious areas that need improvement; it should be a solid piece that can simply be refined through an A/B test.

Create the variant, testing ONE variable at a time.

Now, with the control selected, it’s time to create the variant. In the variant, you can test the creative components of the mailer or the audience.

There are a few ways you can test the creative components of a mailer. For example, say you are testing a coupon code for a $20 car wash. You could test whether an offer of “$5 off” or “25% off” works better. Or, you could test whether a photo of a dirty car or a photo of a clean car generates more responses.

You could also send marketing postcards with the same creative to two different age groups, for example, to see which age bracket has a higher response rate. And so on. No matter what you do, it’s critical to test only one variable at a time. Testing numerous variables at once will make it impossible to definitively name the “winning” mailer.

Ensure that the success of both mailers can be uniquely tracked.

If you’ve selected a specific number of phone calls as the success metric for your A/B test, you’ll need to use a different phone number for each version of the mailer so you can easily track which variant was more successful. If you’re tracking website visits, use two different URLs. If you’re tracking the usage of an in-store coupon, send the coupon offers in two different colors. Without unique tracking, you’ll have no way of determining which campaign garnered better results.

Send out your campaign to the right sample.

We recommend using only a portion of your mail list for the A/B test — rather than sending the test to your entire audience at once. So, if you have a mail list of 500 people, send the A/B test to a group of 100 of those people first and see how your sample responds. Once a winner is determined, you can roll it out to your entire list.

Track the results and keep optimizing.

Now, you’re ready to send out your A/B test! Track results compared to the goal you set over a period of approximately one month. This will give your mailer plenty of time “in the field.” Once the month has passed, tally the results and optimize your campaign according to the winning mailer.

Rarely, however, will one test be adequate. While there is no such thing as perfection in marketing, A/B testing allows you to get a much better idea of what’s working. Test multiple variables until you are confident that your campaign is as optimized as possible, and then continue to test campaigns in the future. Your success metrics will thank you for it!

If you need a trusted storefront to print your direct mail A/B test materials, we’ve got you covered at QuantumPostcards. Just check out our home page to find the right printing and postcard mailing solution for your needs.