Free Lesson

Pressure-test any AI analysis

Part of The AI Evaluation Handbook

60 min
Jun 24, 2026 11:00 AM
Virtual (Zoom)

In this video

What you'll learn

Find or forge a reference to check an AI analysis against

The spectrum, and which fits your company

Handle the novel question that has no answer key

With checks you can run on any output

Know what each check actually catches

And where it falsely passes

Join AI Builders Slack Community. Link: bit.ly/ai-connect

Learn Agentic Analytics workflows and AI Eval design. Get feedback on what you’re building from other AI builders.

Decide when to act, when to dig,

And when to abstain

Why this topic matters

AI analytics tools are confident even when they're wrong, and "is this right?" is the question they all raise. Knowing means checking against ground truth. Sometimes you have a reference (there's a spectrum of ways to get one, and which fits depends on your company), and sometimes you genuinely don't (the novel question, which is exactly where AI is most useful). This lesson covers both.

You'll learn from

Shane Butler

Shane Butler

Co-Founder AI Analsyt Lab (Ex-Stripe)

Sravya Madipalli

Sravya Madipalli

Senior DS Leader (Ex-Microsoft)

Hai Guan

Hai Guan

Head of Data at Ontra, Ex-LinkedIn

Stripe
Nextdoor
eBay
Microsoft
LinkedIn
See all products from AI