AI Evals for Claude Code Analytics Output

Part of The AI Evaluation Handbook

Hosted by Shane Butler, Sravya Madipalli, and Hai Guan

Wed, Jun 17, 2026

3:00 PM UTC (1 hour)

Virtual (Zoom)

Free to join

1,060 students

Invite your network

Go deeper with a course

Build Agentic Analytics 101 in Claude Code
Shane Butler, Sravya Madipalli, and Hai Guan
View syllabus

What you'll learn

See how an agentic analysis actually breaks

The path from question to SQL to recommendation, and where confident-but-wrong answers slip in

Learn the variety of ways to check any AI analysis

and how to match the effort to the stakes

Watch an AI Eval reliability test live

Run a question several times and measure what's stable vs what drifts (no answer key required)

Walk away with a one-page checklist you can use on Monday

Something you can immediately apply to you day-to-day

Why this topic matters

AI will now run your whole analysis in seconds, confident, polished, and sometimes completely wrong. Getting the answer stopped being the hard part; knowing whether to trust it is the new skill. This session teaches the high-level framework for validating agentic analytics from scratch: what it is, where it breaks, and the few ways to check any output.

You'll learn from

Shane Butler

Principal Data Scientist at Ontra

Shane Butler is a Co-Founder of the AI Analyst Lab. Previously he led evaluation strategy for AI product development in the legal tech domain. He has more than ten years of experience in product data science and causal inference, with prior roles at Stripe, Nextdoor, and PwC. Shane is also the co-host of the AI podcast Data Neighbor, where he interviews product, data, and engineering leaders who are pioneering the next generation of data science and analytics in an AI-driven landscape.

Sravya Madipalli

Senior DS Leader (Ex-Microsoft)

Sravya Madipalli is a Senior Manager of Data Science with 14+ years of experience helping teams make better decisions with data. She has built and led data science and product analytics teams at Microsoft, eBay, Nextdoor, and Superhuman (prev. Grammarly), working closely with product, engineering, marketing, and leadership. Her expertise spans experimentation, metrics design, modeling, analytics, and translating complex user behavior into clear, actionable insights.

Hai Guan

Head of Data at Ontra, Ex-LinkedIn

Hai Guan leads the data organization at Ontra, the leading legal tech AI solutions for private markets. He previously led Data Science & Analytics at LinkedIn, Nextdoor, Pinterest, and Meta. He's spent a decade teaching product development teams how to ask questions that actually drive decisions—and now teaches how to combine that judgment with AI to move 10x faster.

Previously at Stripe, Nextdoor, PwC

Stripe
Nextdoor
Ontra
PwC India
AppFolio
See all products from AI Analyst Lab

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.