AI Evals for Product Managers

Hosted by Anshumani Ruddra

Thu, Apr 23, 2026

11:00 PM UTC (1 hour)

Virtual (Zoom)

Free to join

74 students

Invite your network

Go deeper with a course

Build, Evaluate and Ship Agent Skills: Your AI's Superpower
Anshumani Ruddra
View syllabus

What you'll learn

Own & Lead AI Evaluation as the PM

Understand why the PM is the ideal AI evaluation owner. Use empathy and product context to define performance criteria

Build a Robust Generalizable AI Evaluation Framework

Move from one user to diverse data, use a 'living' Golden Set for regression, and utilize metrics beyond just accuracy

Implement Continuous Evaluation and Avoid Pitfalls

Automate evaluation as a continuous process. Use LLMs for qualitative checks and avoid pitfalls

Why this topic matters

Most AI features fail not because of bad models, but because of bad evaluation. Production AI systems need systematic evaluation - and PMs need to understand how to spec it, measure it, and improve it. You'll learn the eval loop: define "good" -> build your Golden Set -> choose your eval type -> automate & iterate

You'll learn from

Anshumani Ruddra

Product Leader and Super IC at Google

I have spent the last 22+ years crafting experiences and products for users: first as an author of children’s books, then as a game designer on some of the world’s largest social games and then as an entrepreneur and product leader across various global consumer tech businesses (gaming, messaging, healthcare, education, media and payments).

Across large companies and early and mid stage startups, I have managed products (multiple 100M+ monthly users) and teams of varying sizes. At Google, I helped build one of the largest AI-powered customer help and support chatbots on one of the world's most popular payments' app.

Google
Disney+ Hotstar
Zynga

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.