Photo source: Airbnb
From online and offline to digital and traditional, marketers have a lot of places where they can spend their dollars. Figuring out the right channels to invest in, however, can pose a lot of questions, most of which can be answered by looking at data and performance metrics.
During a Facebook Measurement Event held at the social network’s New York office, Facebook’s VP of global marketing solutions Carolyn Everson discussed performance measurement trends with customers PepsiCo and Airbnb. Here are five key performance measurement takeaways based on the insights from Airbnb’s data science manager Alok Gupta.
1. Justify your marketing spend with data…and be upfront when you can’t
When it comes to determining which channels to invest in, Gupta said there’s only one way to know for sure: test and measure. “We will only spend where we are able to measure,” he said.
While measuring digital channels like email and display has gotten easier for marketers, many still struggle with tracking the effectiveness of traditional channels, like billboards or television. Gupta still views these traditional channels as “strategic” and said Airbnb does try to run tests on these channels and invest in them. However, he said that he’s upfront in terms of expressing these channels’ measurement limitations, especially when comparing traditional channels’ performance metrics to those of digital channels.
“We’ll be realistic about the decisions that need to be made,” he said.
2. Understand that the lines between brand marketing and direct marketing are blurring
Brand marketing and direct marketing can sometimes be viewed as separate. One focuses on top-of-funnel metrics (e.g. awareness) while the other focuses on bottom-of-the-funnel metrics (e.g. conversion). However, Gupta said that the divisions between these two areas are beginning to dissolve.
“We do have some notion of advertising or marketing for the purpose of brand awareness over the long term and a strategy for performance or D.R. advertising for conversions in the short term that we would like to measure,” he said. “But, having said that, these lines are getting very, very blurry.”
3. Adopt a systematic test-and-learn mentality and experiment often
Airbnb’s data science team works with marketing to take a “systematic approach to evergreen experimentation,” Gupta said, which includes creating hypotheses, testing them rapidly, and then rejecting or confirming them.
In one Airbnb test, for instance, Gupta and his team tried to determine whether a piece of creative targeting new users would actually generate a return on investment. They hypothesized that the creative would give new users with high intent the nudge they needed to convert. So, the team ran a test and showed the creative to some of its new users. While Gupta said the team saw some conversion among new users, he admitted that the amount of money generated from this creative was not as much as the brand had spent. As a result, the ROI was negative. However, he and his team identified a “sweet spot of intent.” When they targeted this specific group, the brand was able to spend “just enough dollars” to produce a positive ROI.
“It’s called marketing science for a reason,” Gupta said in regards to his experimentation philosophy.
Airbnb applied this same kind of testing to its highly talked about logo rebrand in 2014. Gupta said that the team ran focus groups before the launch and that the brand had seen a consistent initial reaction. However, he admitted that there were some “fun reactions” after the logo debuted that were not anticipated (see here). Still, he claims that the logo was “generally well received” and “in line” with the reactions from Airbnb’s focus groups.
4. Be open to collaboration and be honest with your partners
Airbnb’s data science team works with marketing throughout every point in the purchase funnel, Gupta said. The brand also works with third-party vendors, like Facebook.
While Gupta said Facebook has been “very receptive” to the brand’s needs, he admitted that, as an advertiser, there’s always more Airbnb would like Facebook to do. For instance, he said that he’d like Facebook to do more in terms of data logging and customization—such as being able to see more household data versus individualized data since most people in the same household go on vacation together.
“The key message here is [the requests are] never going to end,” he said.
5. Know that measurement mistakes will happen.
Facebook made headlines in 2016 for miscalculating its average duration of video viewed metric. However, Gupta said that as long as bugs are identified quickly, corrected thoughtfully, and happen infrequently, that’s really the best advertisers can expect. As he put it, “We’re never going to build the perfect system ever.”
