Free Lesson

AI Evals for Claude Code Analytics Output

Part of The AI Evaluation Handbook

60 min
Jun 17, 2026 11:00 AM
Virtual (Zoom)

In this video

What you'll learn

See how an agentic analysis actually breaks

The path from question to SQL to recommendation, and where confident-but-wrong answers slip in

Learn the variety of ways to check any AI analysis

and how to match the effort to the stakes

Watch an AI Eval reliability test live

Run a question several times and measure what's stable vs what drifts (no answer key required)

Walk away with a one-page checklist you can use on Monday

Something you can immediately apply to you day-to-day

Why this topic matters

AI will now run your whole analysis in seconds, confident, polished, and sometimes completely wrong. Getting the answer stopped being the hard part; knowing whether to trust it is the new skill. This session teaches the high-level framework for validating agentic analytics from scratch: what it is, where it breaks, and the few ways to check any output.

You'll learn from

Shane Butler

Shane Butler

Principal Data Scientist at Ontra

Sravya Madipalli

Sravya Madipalli

Senior DS Leader (Ex-Microsoft)

Hai Guan

Hai Guan

Head of Data at Ontra, Ex-LinkedIn

Previously at Stripe, Nextdoor, PwC

Stripe
Nextdoor
Ontra
PwC India
AppFolio
See all products from AI