15 Advanced & Out-of-the-Box Ways to Use Incrementality Testing in Marketing

Let’s be real: figuring out what actually works in your marketing campaigns can be a real headache. But incrementality testing? It’s a game-changer. It helps you cut through the noise and figure out what really drives results. Here are 15 advanced and creative ways to use incrementality testing, each with practical examples.

Why Incrementality Testing Matters

Before we dive into specific tactics, let's talk about why incrementality testing is crucial. Simply put, it helps you understand if your marketing efforts are driving additional results beyond what would have happened anyway. This can have a massive impact on your ROI and overall marketing strategy.

1. Optimal Timing and Frequency for Remarketing

Ever wonder if you’re showing your ads too often or not enough? Use incrementality testing to find the perfect timing and frequency. For instance, you can have one test group that sees your ads every two days and another that sees them every five days. The control group wouldn’t see any remarketing ads. The success metric here would be the conversion rate, allowing you to find the sweet spot that maximizes sales without annoying your audience.

2. Testing New Customer Bidding in Google Ads

Curious if using the "only bid for new customers" setting in Google Ads leads to more incremental revenue? Create a test group where the setting is enabled and a control group where the campaign bids on both new and existing customers. Compare the incremental revenue generated by each group. The success metric would be the additional revenue from new customers versus the cost of acquiring them.

3. Experimenting with Ad Creatives and Messaging

Bored of the same old ad creative? So are your customers. Experiment with different ad creatives and messages. Have one test group see a humorous ad and another see a straightforward sales pitch, while the control group sees no ads at all. The success metric here would be the lift in conversion rates, helping you determine which creative resonates best.

4. Measuring Advertising Impact on New Product Launches

Launching a new product is always a gamble. Use incrementality testing to measure the true impact of your advertising. Have one group exposed to your new product ads (test group) and another group that isn’t (control group). The success metric is the incremental sales generated by the new product ads, showing if your marketing efforts are driving additional purchases or just shifting sales from existing products.

5. Adjusting Targeting and Channels for New Products

Fine-tuning your targeting and media mix can be tricky. Use incrementality testing to identify which audiences and channels generate the most lift for your new product. For example, have one test group targeted through social media ads and another through search ads, with a control group that sees no new product ads. The success metric is the incremental sales generated by each channel, helping you allocate your budget more effectively.

6. Tailoring Messaging for New Products

Your messaging needs to hit the mark, especially for new products. Run tests with different messages. Have one test group see ads with a feature-focused message and another with a benefit-focused message, while the control group sees no new product ads. The success metric would be the lift in engagement and conversion rates, allowing you to refine your approach based on what resonates most.

7. Always-On Incrementality Measurement

Set up an always-on incrementality testing framework to continuously measure the impact of your paid media campaigns. Maintain a control group that isn’t exposed to any of your ongoing campaigns, while the test group is exposed. The success metric here is the continuous measurement of incremental sales, allowing you to make real-time adjustments to your campaigns.

8. Constructing Synthetic Control Groups

With always-on measurement, synthetic control groups become essential. Use impression data and household graph data to construct these groups. For example, if your test group is exposed to a new ad campaign, create a synthetic control group using data from households that have similar characteristics but weren’t exposed to the campaign. The success metric is the difference in sales between the synthetic control group and the test group, providing a clear picture of your campaign’s incremental impact.

9. Trend Analysis Over Time

Analyze incrementality results over time to identify trends. Have a test group that is continuously exposed to your campaigns and a control group that isn’t. The success metric would be the trend in incremental sales over time, allowing you to identify patterns and make data-driven decisions.

10. Measuring Different Funnel Stages

Incrementality testing isn’t just for conversions. Measure the incremental impact at different stages of the funnel. For example, have one test group see awareness ads and another see conversion ads, with a control group that sees no ads. The success metrics are the lift in brand awareness for the awareness group and the lift in conversions for the conversion group, helping you understand how your campaigns contribute at each stage.

11. Upper vs. Lower Funnel Campaigns

Test your upper funnel brand campaigns separately from your lower funnel direct response campaigns. Have one test group exposed to brand awareness ads and another to direct response ads, with a control group seeing no ads. The success metric would be the incremental lift in brand awareness for the upper funnel and conversions for the lower funnel, allowing you to allocate your budget more effectively.

12. Channel Contribution at Each Funnel Stage

Different channels contribute to various stages of the funnel. Use incrementality testing to identify which channels drive awareness, consideration, and conversions. For example, have test groups exposed to social media, search, and email campaigns, with control groups seeing no ads. The success metrics would be the incremental lift in awareness, consideration, and conversions for each channel, helping you adjust your media mix for maximum impact.

13. Measuring Offline Channel Impact

Don’t forget your offline channels. Apply incrementality testing to measure the impact of TV and radio ads. Use geo-holdout tests with one group of regions seeing your ads (test group) and another group not seeing them (control group). The success metric is the incremental sales in the regions exposed to your ads, providing insight into the true value of your offline investments.

14. Geo-Holdout Tests

Geo-holdout tests are a powerful tool for offline channels. Isolate specific geographic areas to accurately measure the incremental impact of your campaigns. For example, run a TV campaign in one region (test group) while keeping another similar region ad-free (control group). The success metric is the difference in sales between the two regions, giving you clear insights into how different markets respond to your ads.

15. Cross-Channel Optimization

Analyze the incremental impact of your offline channels alongside your digital efforts. Have test groups exposed to combined online and offline campaigns and control groups exposed to only online or only offline campaigns. The success metric is the incremental lift in sales from the combined efforts, helping you optimize your overall media mix for the most significant results.

Types of Incrementality Tests

Geo Holdouts

Isolate geographic areas to measure the impact of your campaigns. Great for testing offline channels like TV and radio. The test group would be regions where the ads are shown, and the control group would be regions without ads. The success metric is the incremental sales in the test regions.

Inverse Holdouts

Turn off certain campaigns to see what happens to your sales. Perfect for understanding the true impact of your prospecting or retargeting efforts. The test group would have the ads turned off, while the control group would continue seeing them. The success metric is the change in sales when the ads are off.

Scale Tests

Increase or decrease your ad spend in certain areas to measure the incremental lift. Use this to find the optimal budget allocation. The test group would have increased or decreased spend, while the control group would maintain the original budget. The success metric is the difference in sales due to the budget change.

Branded Holdouts

Turn off branded search campaigns to see how much they contribute to your overall sales. This helps you understand the true value of your branded efforts. The test group would have the branded campaigns turned off, and the control group would continue with them. The success metric is the sales difference when the branded campaigns are off.

Unconventional Incrementality Testing Approaches

Regional Email Holdout Test

Segment your email list by region and hold out retention emails in certain areas. This can reveal the incremental impact of your email marketing campaigns on repeat purchases, allowing you to measure how much your email efforts truly drive additional orders versus what would have happened organically.

Product Bundle Incrementality

Offer product bundles or package deals to a subset of customers and present individual products at regular prices to another subset. Compare the average order value and purchase frequency between groups to determine if bundling strategies incrementally boost sales and customer lifetime value.

Loyalty Program Tier Testing

Randomly assign customers to different loyalty program tiers regardless of their actual spending levels:

• Test Group A: Premium tier benefits.
• Test Group B: Standard tier benefits.
• Control Group: No loyalty program.

Measure the incremental impact of loyalty tiers on purchase behavior and customer retention over time.

Personalization Algorithm Effectiveness

Expose users to product recommendations generated by your personalization algorithm in the test group, while the control group sees generic or randomized product suggestions. This test can reveal the true incremental value of your personalization efforts in driving conversions and average order value.

Customer Service Proactivity

Proactively reach out to customers with personalized support or product usage tips in the test group, while the control group only receives reactive support. Measure the incremental impact on customer satisfaction, retention, and upsells to quantify the value of proactive customer service initiatives.

Social Proof Messaging

Display real-time social proof messages (e.g., "10 people are viewing this item") on product pages for the test group, and show product pages without social proof elements to the control group. This test can reveal the incremental impact of social proof on conversion rates and average order value.

Free Shipping Threshold Testing

Randomly assign customers to different free shipping thresholds:

• Test Group A: $50 minimum for free shipping.
• Test Group B: $75 minimum for free shipping.
• Control Group: No free shipping offer.

Measure the incremental impact on average order value and overall revenue to optimize your free shipping strategy.

Offline Incrementality Testing Methods

Geographic Testing

Select comparable regions or markets, run the offline campaign (e.g., print ads, events) in some areas but not others, and compare key metrics like sales, foot traffic, or brand awareness between test and control regions. This allows you to measure the incremental lift provided by the offline campaign.

Time-Based Testing

For channels like events or seasonal promotions, measure performance metrics before, during, and after the campaign period. Compare to historical data and account for seasonality, analyzing the incremental impact during the campaign window.

Matched Market Testing

Identify similar markets or regions, run the offline campaign in one market but not the other, and compare performance between the matched markets to isolate campaign impact.

Surveys and Brand Lift Studies

For brand-focused offline campaigns, conduct surveys before and after the campaign to measure changes in brand awareness, perception, and purchase intent. Compare results between exposed and unexposed audiences.

Key Takeaways for Marketers

• Define Clear Objectives: Know what you’re trying to achieve with your tests.
• Choose the Right Metrics: Focus on metrics that align with your business goals.
• Maintain Control Groups: Ensure your control groups are comparable to your test groups.
• Analyze and Act: Use the insights gained to optimize your campaigns and improve ROI.

Conclusion

Incrementality testing is a powerful tool that can help you optimize your marketing efforts and drive better results. By understanding what truly works, you can make data-driven decisions that boost your ROI and ensure your marketing dollars are well spent. Whether you’re adjusting ad creatives, targeting new audiences, or measuring offline impact, incrementality testing provides the insights you need to succeed.

How Stella Can Help

Stella makes incrementality testing easy. Our platform is designed to help you conduct, analyze, and take action on your incrementality tests without the hassle. Whether you're optimizing remarketing campaigns, launching new products, or measuring offline impact, Stella's tools provide the insights you need to drive the most incremental results.

Ready to supercharge your marketing with always-on incrementality testing? Try Stella today and discover how easy it is to unlock the full potential of your campaigns.