Measure AI Impact with Causal Inference

Hosted by Shane Butler

Wed, Feb 4, 2026

8:00 PM UTC (1 hour)

Virtual (Zoom)

Free to join

Invite your network

Go deeper with a course

AI Analytics for Builders
Shane Butler, Sravya Madipalli, and Hai Guan
View syllabus

What you'll learn

Know when you need causal inference

Use it when you cannot randomize but still need decision-grade evidence for product or business impact.

Choose among three practical methods

Panel regression, diff-in-diff, and propensity matching: what each assumes and when each is appropriate.

Stress-test credibility before you trust it

Learn simple checks to spot confounding risk and communicate uncertainty without hand-waving.

Why this topic matters

Many AI launches cannot be cleanly randomized, yet teams still need credible impact estimates. Causal inference can be stronger than correlation, but only if the assumptions are clear. This lesson helps you choose an approach, understand what must be true, and avoid trusting numbers you cannot defend.

You'll learn from

Shane Butler

Principal Data Scientist, AI Evaluations at Ontra

Shane Butler is a Principal Data Scientist at Ontra, where he leads evaluation strategy for AI product development in the legal tech domain. He has more than ten years of experience in product data science and causal inference, with prior roles at Stripe, Nextdoor, and PwC. His current work focuses on practical, end-to-end methods for evaluating AI features in production. Shane is also the co-host of the AI podcast Data Neighbor, where he interviews product, data, and engineering leaders who are pioneering the next generation of data science and analytics in an AI-driven landscape.

Previously at Stripe, Nextdoor, PwC

Stripe
Nextdoor
PwC India
Ontra
AppFolio

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.