One of the topics I’m often asked about by early-stage startups is whether it’s possible to test into TV at smaller budgets, and if so, is it even measurable?
The answer to both is a resounding: YES!
While the methods of buying TV media remain antiquated relative to newer digital channels, this doesn’t mean the channel is inaccessible or unmeasurable. Following are a few reasons why you might consider adding TV to your media mix, and how you might approach measurement.
Note: the below assumes you’re advertising with the purpose of driving performance. If you’re much later-stage and hold an objective of driving awareness and brand-building, then some of this might not apply. However, I would argue that both brand and performance ladder up to the same objective, just on different time horizons.
Tactic: spike (or "burst”) analysis
Assuming you’re buying national media, which often comes at a fraction of the cost per impression (CPM) of regional media, one common method of measuring TV is spike, or burst, analysis. This is a relatively straightforward process:
- Collect post-logs. The TV buying agency provides the post-log, which details the exact time (down to minute) a spot aired.
- Establish a baseline to account for any potential seasonality or time-of-day trends and map a baseline of traffic (or signups) to the most granular practical level. Ideally, time of year –> day of week –> time of day.
- Measure lift above baseline for a [x] minute window after each spot airs, measure the lift in [traffic/signups] above baseline. This lift gets attributed back to TV.
- Calculate cost per acquisition (CPA): using the method above, aggregate the data to a level that makes sense for your business — weekly CPA, spot-level CPA, network-level CPA, daypart CPA, etc.
Note in #3 above, you might want to measure lift in the conversion event that makes most sense to your business. Generally this is something other than traffic, such as signups or first purchases. This adds a layer of complexity onto the analysis, as you’ll either need to use conversion assumptions (non-ideal) or track conversion rate from the specific cohort of traffic that arrived during your TV attribution window. The latter is more ideal, but less straightforward and noisy if there is limited data.
The method above provides an estimate of the direct response impact of TV. While this is less useful in comparing the impact of TV to other digital channels (more on that later), it can help optimize within your TV budget, as you compare spot-to-spot, network-to-network, and more.
Tactic: incrementality (“lift”) testing
A more accurate method — not just for offline but for any channel — is to run an incrementality test. There are several methods of going about this, but all follow a similar principle of comparing a control group to an exposed group and measuring total lift in conversions.
For TV, this type of test comes at a significant cost, as it will typically require buying regional media at CPM multiples of national media. However, the cost of not understanding incrementality can end up being much more expensive. Following is a common, simplified process used to conduct a TV lift test:
- Allow for a cooling period. Assuming your brand has been on TV for some time, you’ll want a “dark period” during which no TV media is running. This is because TV has a tail effect, which can last for weeks (or more) after an ad airs. The tail is more likely to be longer-duration the longer you’ve been on air. There is no one-size-fits-all answer on how long this cooling period should be. I highly recommend collaborating with your data science team – but generally, the cooling period is measured in weeks vs months.
- Determine match markets. In this test, you’ll measure lift by buying regional media and comparing exposed markets to control markets. To ensure meaningful insights, you’ll need to pick markets that most closely behave to one another. Typically, match markets are selected based on population size and your brand’s penetration in each of those markets — but you might also look at factors like income, education levels, political lean, conversion rates, etc. It will also help to have a representative sample of small, medium and large markets for the test.
- Hold all other regional efforts constant. To properly control for external factors, you’ll want to make sure test and control groups have the same likelihood of being exposed to other media during the test period. This means you should hold all other channels as steady as possible. You don’t need to turn off ads on other channels (though for measurement purity, it can help) — but you should ensure no regional efforts on other channels would pollute the test.
- Buy regional media. Run your TV campaign as you would nationally. Your agency can help match gross rating point (GRP) levels across each of your test markets.
- Compare exposed vs control market to measure percent lift. You can calculate percent lift in conversions (again, the conversion event that makes most sense to your business) by comparing test vs control markets during your test period. Typically, 4 weeks.
- Translate regional CPMs to national CPMs. For CPA calculations, you’ll need to make assumptions on what your CPM would have been had you been buying nationally. Your CPA calculation will be simple estimation of this spend level over the lift in conversions across test markets.
Assuming you run incrementality tests across digital channels, this method can help you get a more meaningful read on TV’s total impact to your performance. If you want to get more advanced, it can also help you understand the interactions between TV and other channels like SEM, FB, etc — which can help you run a more optimal mix.
So should I think about testing TV?
There is no one-size-fits-all answer to this. When it’s working, TV can create lift not only to your business, but to conversion rates across all other channels (by “priming” those who see impressions across other performance channels). However, the answer is entirely dependent on your core demographic.
Another consideration: this is one area in which going through an agency tends to be valuable, if not necessary. While programmatic and OTT technologies are making advances, the method of accessing linear inventory through networks is still fairly opaque, so you’ll want a partner who has experience (and relationships) in the marketplace.
If you can tolerate swings in spend levels, and can afford a passable spot (30s or 15s) to air on TV, then buying remnant media with a partner like Tatari or Opus Growth can be a valuable test. If your budgets are slightly larger, there are a slew of more traditional media agencies (e.g. Horizon Next) who are beginning to operate with more of a performance lean.
So yes, the space is slightly more intimidating to navigate than the increasingly-automated FB and AdWords ecosystems — but there is still plenty of opportunity for performance advertisers to drive growth in a measurable way.