Like the fit of a shoe, direct mail is not a one-size-fits all medium. From postcards and coupons to flyers and booklets, there are a number of formats and creative options marketers can choose from—each at different price points.
Super Shoes, a footwear retailer specializing in occupational products, used postcards and tri-fold mailers to drive brand awareness and engagement; however, its director of marketing Matt Willard wanted to know if one format worked better than the other. The only problem is Willard knew he didn’t have the resources to run a multivariable test.
“[The direct mail pieces] had varying prices, varying design sets associated with them,” he says. “For us to determine if one was better than the other was going to require multiple events, lots of calculation, and it would become very expensive and very time consuming to try to perform that.”
So in fall 2015, Willard decided to leverage Quad/Graphics’ (Quad Graphics) Applied Marketing Test Process (AMTP)—a testing platform that would allow the footwear brand to analyze multiple combinations of variables.
Sizing up its audience
For the test, Willard identified a few variables within Super Shoes’ direct mail pieces that he wanted to test for performance. For instance, he tested whether customers responded better to postcard or tri-fold mailers and whether they preferred minimalist designs or product-oriented ones. The brand tested these variables by way of online surveys, which was sent to a pool of customers who had opted in to participate. The brand then qualified that list by geographic and demographic data to create a pool that more closely resembled its customer base.
After participants completed the test, Willard worked with Quad Graphics to analyze the findings. Based on the data, the AMTP platform predicted that the postcard would generate a higher ROI—a pleasant forecast considering that the postcard is the less expensive format. But because the postcard is smaller than the tri-fold mailer, Willard was concerned that the postcard would limit Super Shoes’ exposure.
So, Willard decided to run another test in December 2015 to confirm the AMTP predictions.
Strapping on the segmentation
Unlike the first test, which was strictly digital, Willard and his team decided to actually send the postcards and tri-fold mailers to Super Shoes’ consumer base. But not everyone received the same creative.
In the past, Super Shoes had done its own internal segmentation and then sent its analysis to a third party to have them develop personas. The footwear brand decided to tweak the mailers’ creative based on its top-performing and largest persona groups: its influencers, who are more rule-binding and information-hungry, and its trailblazers, who are more individualistic, indulgent, and excited by new ideas, Willard says. In addition to sending the targeted mailers to these groups, Super Shoes had a control group.
Tying up the results
When it came to measuring success, Willard and his team tracked response rate (by way of coupon redemption), as well as sales versus marketing investment. The sales versus marketing investment metric was calculated through a “loose” attribution study, he says, in which Super Shoes analyzed whether the recipients of the mailers made a purchase within the following four weeks.
In terms of response rate, the tri-fold mailer outperformed the postcard with a variance of nearly two points, Willard says. More specifically, the influencer postcard had the lowest redemption rate at 3.5% while the trailblazer tri-fold card had the highest redemption rate at 5.4%. Still, Willard considered this metric more of a “superficial analysis.”
“Redemption rate only gives you the coupons that were redeemed,” he says. “It doesn’t necessarily speak to the amount of money being spent or the quality of those respondents.”
As for sales versus marketing investment, Willard says this metric was “extraordinarily higher” for both the influencer and trailblazer postcard, confirming the AMTP prediction. In fact, the tri-fold mailer and the postcard outperformed Super Shoes’ baseline by 200%.
Polishing up the strategy
Truly eliminating variance within the 2015 test was one of Super Shoes’ biggest challenges, Willard says. He also admits that it was hard to interpret the results at first with an open mind.
“You go into a review with a set of expectations,” he says, “and it was easy to get caught up in the uncertainty of what was important between response rate versus redemption rate, conversion rate and marketing investment.”
And if there’s one thing he’s learned from this experience, it’s that “not every event was created equal.” In other words, Willard’s goal for a direct mail piece may not be to drive sales all of the time. If he wants to drive awareness and response, for instance, he and his team might send out a tri-fold mailing; however, if the goal is to increase sales, then they may send out a postcard.
“We’re not always seeking to drive increased profit,” he says. “In some cases, we’re willing to settle for a reduced profit for more eyeballs….[AMTP] predicted that the postcard would be more profitable and that the tri-fold would have a greater response. So as we move forward in 2016, we’ve adopted those sensibilities into our direct mail strategy.”
Super Shoes has continued to work with Quad Graphics on several print and digital initiatives. For instance, the companies ran a mobile campaign in which customers within a certain radius of a Super Shoes’ store would receive an ad directing them to a newly created mobile site. In addition, the companies recently completed a print campaign in which a fall catalog was sent to a large portion of Super Shoes’ customers. When customers engaged with small icons inside of the catalog, they could view additional content, like behind-the-scenes video from a photo shoot.
