Free Lesson
Pressure-test any AI analysis
Part of The AI Evaluation Handbook
60 min
Jun 24, 2026 11:00 AM
Virtual (Zoom)
In this video
What you'll learn
Find or forge a reference to check an AI analysis against
The spectrum, and which fits your company
Handle the novel question that has no answer key
With checks you can run on any output
Know what each check actually catches
And where it falsely passes
Join AI Builders Slack Community. Link: bit.ly/ai-connect
Learn Agentic Analytics workflows and AI Eval design. Get feedback on what you’re building from other AI builders.
Decide when to act, when to dig,
And when to abstain
Why this topic matters
AI analytics tools are confident even when they're wrong, and "is this right?" is the question they all raise. Knowing means checking against ground truth. Sometimes you have a reference (there's a spectrum of ways to get one, and which fits depends on your company), and sometimes you genuinely don't (the novel question, which is exactly where AI is most useful). This lesson covers both.
You'll learn from

Shane Butler
Co-Founder AI Analsyt Lab (Ex-Stripe)

Sravya Madipalli
Senior DS Leader (Ex-Microsoft)

Hai Guan
Head of Data at Ontra, Ex-LinkedIn