The Right Way To Measure Media

Daniel E. Aks, Brian Pozesky and Paul H. Van Wert

THE PROBLEM WITH MEDIA MEASUREMENT

“Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” The quote, from retail magnate and marketing pioneer John Wanamaker, is over 100 years old. Despite digital media’s promise of accountability, many retailers still struggle with this attribution conundrum.

The goal of understanding how marketing and advertising resources are consumed and determining the return on those efforts is necessary to optimizing the marketing mix. Attribution model after attribution model is developed, with endless amounts of touch point data attempting to determine the magic ingredients that drove consumer behavior. Using the wrong models and approaches can lead to exactly the wrong conclusions, thereby being precisely wrong rather than vaguely right. Many are waiting for the perfect model. The situation seems to closely parallel Waiting for Godot, and for those who implore more rationality in media measurement, bringing a rope in the final scene echo as well!

We’ve written this paper because we’ve witnessed first-hand the misleading results of ham-fisted and sometimes lazy models. At Undertone, we offer unique High Impact digital circulars, recipe ads, and more, all personalized through a slew of AI- driven selected variables that drive sales lifts leading to 15x to 19x ROAS. These state-of-the-art products can stymie old
and tired media models. This is because, in some retailer attribution models, high impact display is treated with the same modeling considerations as boring, small and entirely missable standard display ads. Of course, this is plain wrong, yields misleading results and is hurtful to retailer aspirations.

This note attempts to better explain what drives retail store visits, and perhaps a simpler and more effective means of being largely right in motivating people to make a trip to the local grocer, department, or specialty store.

Let’s review first the slew of issues in attribution when using digital channels to encourage physical store visits:

1. Last Touch importance. It’s hard to believe that some modelers still give credence to this model, dismissing or trivializing all prior upper/mid funnel activity that influenced the final touchpoint. Last touchpoints are the results of many initiatives to stimulate desire and exploration. Perhaps useful for very shorter-term, one-time campaigns and products, at best. One-time inventory liquidation might be a good use. Sadly, last touch attribution is the default positioning for most tools, even Google Analytics. It is just preposterous to consider all marketing spend can be attributed to the last link in a strategic chain.
2. First Touch models which believe that simple PPC ads suddenly drive consumer desire! First touchpoints may be relevant for driving to a specific supplier, but desire was created long before. This model completely ignores the effects of other consumer interactions after the initial touchpoint.
3. Multi-Touch, with all its various alternatives (linear, non-direct, customized, time decay, U-shaped)1 modeling has the appealing promise of optimizing media to drive consumers down the purchase funnel. Multi-Touch attribution (MTA) is directionally much better in that at least this modeling attempts to understand that many factors drive behavior, and that multichannel marketing is a proven technique. Still, in the attempt to be precisely wrong, multi-touch models are assigned percentage effectiveness in many cases. Those percentages are highly critical variables and are certainly precise, but almost universally are inaccurate. For example, marketers will develop percentages for video and display, attributing more sales to a video campaign. But what happens when hybrid ad units are used, or dynamic creative is installed, or if both display and video are intertwined strategically? MTA solutions are limited to marketing channels where direct response can be tracked via touch points and consumer identifiers.

In addition, MTA solutions are quickly becoming impossible to implement with the degradation of current consumer identification solutions. Rules based MTA solutions use qualitative judgment and largely touchpoints to determine which channel receives credit, ignoring the full impact of media exposure. Because of the requirements to deploy MTA, media providers often focus on what works to drive strong MTA results, chasing bottom of the funnel touches, reach and/or high frequency, which counterintuitively might not be what is optimal to drive sales. MTA is thus a flawed methodology:

• Association of a sale to a single channel ignores the full value of media in favor of operational convenience. The 99% of exposures that don’t generate touchpoints are significantly undervalued or not valued at all.
• Due to different delivery frequencies, MTA often obscures the value of high impact media formats such as video and rich media.
• Many MTA solutions are not incremental and thus inflate the performance of media by attributing sales that would have happened without media to various marketing efforts.

The final nail in the MTA coffin is the loss of consumer ID solutions as the global privacy movement gains steam. MTA solutions will not be able to track funnel behavior and have large gaps in data collection. Lost signals include, for example:

• Publisher 1st party opt-in requirements
• App Transparency Tracking (ATT) – Apple’s app tracking initiative
• Cookie deprecation (Google’s back-and-forth threats to shut down cookies on Chrome)
• Closed social platforms

For digital marketers, the issue of media effectiveness using newer methods is not just difficult, but getting harder as ID deprecation further scrambles the picture.

INTRODUCING “INCREMENTAL RETURN ON AD SALES, OR iROAS”

It is time for a bit of “back to the future”, and shift from models that try to capture everything happening all at once, and instead do the hard work of determining the incremental sales driven by discrete marketing endeavors. We call this the “iROAS” model. Determining incremental sales relies on the hard work of TEST and CONTROL. This method may not appeal
as something new and sexy; it is as old as the hills. Because each market has some set of unique aspects around competition, income, weather, time of day, and many more exogenous variables, there are no complex yet simultaneously simple-minded holistic models that can predict local retail campaign outcomes. The notion of “iROAS” is to do the painstaking work of holding all media constant market by market, then creating test and control vehicles to isolate the single change in media that can explain sales, traffic or other variable changes.

A NEW APPROACH TO MEASUREMENT

At Undertone, we work closely with Incrementlogy, Inc. to create improved approaches to using media to drive incremental sales. The tents of Incrementology’s approach are:

• Use causation to isolate the full value of media exposure, rather than just touchpoints.
• Deploy machine learning to quickly measure each media channel and calibrate the brand’s entire media portfolio to achieve their goals and implement through an intuitive platform empowers brands to test, measure, and action often to ensure performance.
• Do not depend on consumer identification to measure media value, rather using it for granular insights if/when it is available.

USE POWERFUL MACHINE LEARNING TO MODEL AND SIMULATE OUTCOMES

• Both Channel Measurement & Portfolio Calibration – Use machine learning to quickly measure the full value of individual media channels, as well as providing calibration recommendations for media weight across a brand’s media portfolio.
• Speed to Market – Machine learning replaces the expensive and lengthy integration of solutions such as MTA and media mix modeling (MMM).
• Comparable Metrics Across Channels – Different channels have different performance indicators. Search, display, video and social all have different KPIs. Causal measurement provides the brand with comparable metrics across channels.
• Measurement of Closed Channels – While closed social channels often prevent the identification of consumers to outside media providers, most allow for the use of geography to test for causality. This is something MTA solutions simply cannot provide.

FIND CAUSALITY

• Use machine learning to quickly identify store-level markets (or other geography) with similar sales to identify both test and control markets.
• Causal inference models estimate expected sales in the test markets before the media is launched.
• After media campaigns launch sales in the test and control markets are used to simulate what sales in the test markets would have been without marketing.
• This method does not require consumer level tracking or complex media integration.

CALIBRATE CHANNELS

• Employ machine learning to determine the best calibration or mix of media channels to drive incremental sales.
• Channel calibration can be fine-tuned with inputs from Incrementology’s Incremental Sales Analysis and/or other individual campaign or channel measurement results.
• Using advanced regression techniques and probability distribution methods both the optimal weight and allocation of media channels are determined.
• These same models also provide insight on the effects of each media format across the purchase funnel.

SUMMARY

Media measurement and attribution is a critical tool for retailers, yet deployed in ways that drive the wrong conclusions. It is possible to be both directionally accurate and precise when shifting from naive media measurement effectiveness models to models that focus on incremental sales, executed by identifying causality with advanced machine learning and regression models. The hard work this entails may not sound exciting, but it is what must be done.

ABOUT THE AUTHORS

Daniel Aks is the President of Undertone, A Perion Company, leveraging his broad operating experience in C-level roles for the information, education, & consumer media industries. For the two years before joining Undertone, Daniel was the CEO of Antenna International, where he led their successful turnaround by introducing new products, lowering costs, & building an innovative culture. He also served as COO of the Educational Records Bureau for the prior three years, where he increased revenue by 20%, introduced new assessment products, & reduced costs. Previously, Daniel served as COO/CoS of McGraw-Hill Education where he led its reorganization & developed formative testing & data-driven instruction products. He also served as COO for PRIMEDIA Consumer Magazines & as President of its Internet Business, where he launched & led the pioneering digital business.

Brian Pozesky is the Co-Founder and President of Incrementology. He us an incremental measurement pioneer with 20+ years’ experience building and scaling start-ups. Builder of data science, operations, product and programmatic media groups. Developer and designer of business intelligence tools and large-scale data systems. Holder of patented creative optimization and testing algorithms.

Paul H. Van Wert is the Co-Founder and is the leader of Incrementology’s analytics and Products Marketing capabilities. He is a measurement expert and performance advertising innovator. Experienced in developing retail media networks, leading agile development teams and managing analytics organizations. Product designer at multiple high growth start-ups using cutting edge data & machine learning technologies to deliver strategic insights and accurate measurement of marketing investments.

(1) Source: for a useful overview of many attribution techniques, see https://agencyanalytics.com/blog/marketing- attribution-models#first-interaction-attribution