Free Lesson
Can AI Actually Answer Your Data Questions?
Part of AI Product Engineering
45 min
Jul 15, 2026 3:00 PM
Virtual (Zoom)
In this video
What you'll learn
How to eval data agents
Shreya will show you how simulate the most common failure modes and create high fidelity synthetic data (re: privacy)
How to make data agents reliable
How to provide better structure through tools, workflows and verification strategies.
Common traps when building data agents
Learn the counter-intuitive areas where data agents fail, including how different harnesses and models fail differently.
Why this topic matters
Asking business questions of your data is the hottest enterprise AI use case, and one of the hardest to evaluate. Shreya Shankar & her colleagues built a benchmark that emulates real business scenarios, replicating what trips agents up: scattered systems, messy join keys, answers buried in text. She will share strategies you can use immediately in your workflows to improve performance.
You'll learn from
Shreya Shankar
AI researcher; incoming CS Professor at Carnegie Mellon
Hamel Husain
ML Engineer with 20+ years of experience