Free Lesson

AI Evals for Building Reliable and Consistent Products

Part of Summer of AI Building: Your AI Portfolio in 4 Steps

Hosted by Anshumani Ruddra

Tue, Jul 21, 2026

10:00 PM UTC (1 hour)

Virtual (Zoom)

Free to join

Invite your network

Go deeper with a course

AI Accelerator for Executives and Entrepreneurs
Anshumani Ruddra
View syllabus

What you'll learn

Build a Robust AI Evaluation Framework

Move from one user to diverse data, use a 'living' Golden Set for regression, and utilize metrics beyond just accuracy

Evaluate Different Types of AI Output

Run evals for agent skills, AI products or any kind of system where a generative model is producing an output

Implement Continuous Evaluation and Avoid Pitfalls

Automate evaluation as a continuous process. Use LLMs for qualitative checks and avoid pitfalls

Why this topic matters

Most AI features fail not because of bad models, but because of bad evaluation. Personal and work-related AI systems need systematic evaluation - and AI product builders need to understand how to spec it, measure it, and improve it. You'll learn the eval loop: define "good" -> build your Golden Set -> choose your eval type -> automate & iterate.

You'll learn from

Anshumani Ruddra

Product Leader and Super IC at Google

I have spent the last 22+ years crafting experiences and products for users: first as an author of children’s books, then as a game designer on some of the world’s largest social games and then as an entrepreneur and product leader across various global consumer tech businesses (gaming, messaging, healthcare, education, media and payments).

Across large companies and early and mid stage startups, I have managed products (multiple 100M+ monthly users) and teams of varying sizes. At Google, I helped build one of the largest AI-powered customer help and support chatbots on one of the world's most popular payments' app.

Google
Disney+ Hotstar
Zynga
See all products from Anshumani

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.