An inquisitive approach will uncover opportunities for growth

"While there is no perfect ad measurement solution -- in the same way there is no perfect understanding of what motivates human behavior..." -Gaurav Shirole, Director of Ad Measurement, Roku

In the 1970s, researcher Alan Hedges proposed a radical, if sobering, theory: there is no single test to understand why an ad works. Advertising – and humans, for that matter – are complex systems too hard to universally untangle.

Fast forward to today – ad effectiveness is still a challenge for marketers. In their landmark 2007 paper, “Marketing in the Era of Accountability,” Les Binet and Peter Field found a relationship between brand awareness and business outcomes, but the relationship was too weak to be significant. In 2018, academics at Northwestern University reviewed 15 ad campaigns on a social platform and found half had no impact.

Another study found that for every dollar eBay spent on its search advertising in 2018, the brand lost 63 cents, showing that even large, sophisticated advertisers find room for improvement.

While there is no perfect ad measurement solution -- in the same way there is no perfect understanding of what motivates human behavior – decades of research find that advertising grows brands. Here are four important ways that advertisers can bring a more scientific approach to ad measurement.

Test one, specific hypothesis

Start with the scientific method: develop a specific hypothesis to test. Ad measurement works best when marketers are solving a specific business problem. Maybe too few consumers know about the brand. Perhaps the problem is not awareness, but conversion. Product growth may be strong, but maybe it is unclear which media touchpoint is responsible for the growth. The best campaign measurement does not comb through limitless combinations for learnings; it is designed to test the pre-specified assumptions about the tactics driving a brand’s success. Controlled experiments should focus on incrementality, holding out users who would have seen a spot and matching the exposed group on both demographic and behavioral distributions.

Seek the most accurate consumer signals

Today’s customer journey is fragmented across multiple platforms and channels. The stronger the consumer signal, the more accurate the measurement. Platforms with direct customer relationships hold the foundation for strong and precise identity graphs, which in turn create better control groups, more accurate confidence intervals, and sharper experimentation. These graphs are even stronger when platforms have access to data across devices. Without direct consumer identity, attribution falls into guesswork.

Narrow your bets

Too often, test and learn budgets are not large enough to learn. Consumer habits differ across geographies, age groups, and income levels – they also change over time. A campaign needs enough statistical power to understand how a brand and platform work together. Said another way, if there isn’t enough weight on the scale, an advertiser will not know if results reflect a platform’s capability or lack of investment. It is usually better to place a few, big bets rather than to spread test and learn thin across partners.

Stay curious

The core of the scientific method is reformulating hypotheses based on observation and experimentation. Marketers that take a skeptical and inquisitive approach to their measurement will uncover important opportunities for growth. As with the “hardest” of sciences, genuine curiosity, a problem-solving attitude, and a willingness to question static truths will move us forward.

Roku pioneered streaming to the TV, connecting users to content they love, enabling publishers to build and monetize large audiences and providing advertisers with unique capabilities to engage consumers.