Sophisticated digital advertisers use incrementality testing. This allows advertisers to isolate unique contributions coming from their many digital marketing channels. As marketers move into CTV as the next growth channel, incrementally testing is especially important. CTV is unique in the digital advertising domain and because consumers cannot click on TVs. Incrementality testing is a foundational way for marketers to go “beyond last clicks attribution“. This of course is a requirement when there is no last click.

From defining your audience to calculating incrementality. We will show you how to apply this methodology to the fastest-growing video advertising channel in the world.

What Is Incrementality Testing?

Incrementality testing in advertising is an approach to measure ad performance measurement. It allows marketers to determine the exact level of contribution, or “lift”, that a channel’s advertising has on audience behavior. It requires using two groups: the test group and the control group. The test group is exposed to the advertisement. The control group is not.

If the exposed group performed better, then the test has shown a positive incremental lift. However, if the control group outperformed the exposed group, then the test shows there wasn’t incremental lift. And if there is no difference in the results, the test is deemed neutral. A difference between the results is only meaningful if the gap is statistically significant.  

Incrementality testing is a type of A/B testing. Instead of comparing two advertisements, incrementality testing determines the value of advertising at all. In other words, incrementality testing is about understanding whether advertising on a given channel works. And exactly how much it works.

How to Run Incrementality Testing for CTV Advertising

Step 1: Create a Hypothesis and Establish Performance Metrics

It’s surprising how often advertisers can jump into a testing process before clearly establishing what they’re trying to test. Or how they’re going to assess success. With incrementality testing, you’re testing whether an advertisement works better than not advertising. This means you could test messaging and creative. But you could also test with placebo or dummy ads. These would measure the fundamental value of the channel.

On top of generating your hypothesis, you also want to agree on the performance metrics you’ll be using to evaluate that hypothesis. These are usually metrics like purchases, visits, average order value (AOV), conversion rate (CVR), and visit rate.

Step 2: Define Your Testing Audience

Next you need to build a testing audience derived from your overall target audience. This testing audience will contain the control and the exposed group. When defining this subset, you’ll want to account for any variables that could have an impact on the outcome of your test. For example, if you’re advertising a luxury product, it would likely be a good idea to hold income constant across the groups. Without accounting for these variables, the results could be erroneous. As would be any actions taken as a consequence.

For advertisers looking for precision audience control during testing, our CTV advertising and attribution platform provides numerous targeting parameters. These include age, gender, education, income, household and zip code. In addition, we can target by brand affinity, online purchases, offline purchases, viewing habits, device, business vertical, and fifteen thousand more segments. This level of granularity provides advertisers with complete control over their incrementality testing process.

Step 3: Determine Timeframe and Ad Content, and Run Your Test

The timeframe of the test depends on the volume of interactions, not any specific amount of time. It’s accepted best practice to run your experiment for a minimum of a week. Generally speaking, you should also consider running your test during a relatively normal time of the year. Holidays, elections, and any other major event could impact your audience’s state of mind. This could undermine the validity of your results.

Now, you need to develop ad creative based on the hypothesis you’ve set out to test. If you’re measuring the raw, incremental value of CTV advertising, you need to run placebo creative during your test. This creative needs to be irrelevant to your business. Whether it’s a placebo or a public service announcement (PSA) ad. By running this kind of test campaign, you’ll be able to get an accurate handle on the baseline value of CTV advertising.

With these done, it’s go time. Flight your ad to your test group while suppressing the control group’s exposure.

Step 4: Analyze Test Data and Calculate Incrementality

Once your test is finished, collect all the campaign data from your two groups and review them side by side. From visits to CVR, analyze all your performance metrics. And use them to calculate the incremental lift, as an overall percentage, that the test group saw versus the control group. Using this percentage, you can then calculate the incremental cost per acquisition (iCPA), conversion value, and return on ad spend (iROAS). With these figures in hand, you now have an exact read on the value CTV advertising brings to your business.

Finding a Platform that Supports Incrementality Testing

The impact of traditional TV ads has always been difficult for advertisers to pin down. With the arrival of CTV advertising, those days are behind us. Now, advertisers can measure the value of their ads with the kind of precision that had only been possible on digital channels. They no longer need to wonder if their TV ad campaigns are working. All they need to do is run incrementality testing on their CTV advertising platform to find out.

Advertisers looking for a CTV advertising platform that provides a comprehensive testing toolkit should consider AlphaDigital. We have the world’s first CTV advertising and attribution platform We provide a robust, built-in incrementality testing feature. This feature allows advertisers to run ad hoc tests to assess channel value. And will also run continually to adjust for the lack of ad exposure data via walled garden environments. The only platform on the market with this always-on feature. Our platform was designed to ensure advertisers are consistently getting value out of their CTV campaigns. A commitment we also back up with a fraud-free guarantee.

Reach out to the team to learn more, or get started with your first CTV incrementality test!