Design Evals That Catch Real Failures

Hosted by Aki Wijesundara and Manu Jayawardana

Mon, Aug 31, 2026

9:30 AM UTC (30 minutes)

Virtual (Zoom)

Free to join

Invite your network

Go deeper with a course

AI Engineering Bootcamp and Certificate: Build and Ship Production AI Agents
Dr. Aki Wijesundara and Manu Jayawardana
View syllabus

What you'll learn

Read production traces and cluster the failures

Group real errors by cause so you fix categories rather than one-off symptoms

Turn a cluster into a running assertion

Convert an observed failure into a check that runs on every future change.

Validate an LLM judge before trusting it

Test the grader against human labels so its scores mean something.

Why this topic matters

Most teams are running a vibe check and calling it evaluation. Someone tries a fewprompts, it looks fine, it ships. Then it fails in production in a way nobody thought to test.Real evals are built from actual failure traces, not from imagination, and that changesboth what you measure and how much you trust the result.

You'll learn from

Aki Wijesundara

AI Advisor | Educator | Google AI Accelerator Alum

Aki Wijesundara is an AI leader with a PhD in Machine Learning and extensive experience mentoring startups at Google’s AI Accelerator. With a career spanning both research and applied AI, Aki has taught 5,000+ students worldwide how to design and deploy production-ready AI systems.

He has worked across cutting-edge areas of applied AI, from LangChain and RAG pipelines to observability and large-scale deployment. As a researcher and educator, Aki bridges the gap between theory and practice, making complex systems approachable and actionable for engineers, founders, and product leaders.

Aki is also a frequent speaker and advisor to organizations adopting AI, helping them transition from experimentation to production at scale.

Manu Jayawardana

AI Founder | CEO at TAI Labs | Co-Founder of Snapdrum

Manu Jayawardana is a serial entrepreneur with multiple AI startup successes.

He is also the Co-Founder of Snapdrum, an AI consultancy helping companies integrate and automate their businesses using AI. Through Snapdrum, Manu has worked with top startups and enterprises across the U.S., Europe, and Asia, advising on automation, product development, and AI strategy.

Currently, Manu is the Co-Founder & CEO of Krybe, an AI voice agent startup in London transforming how businesses automate customer interactions. His experience spans fundraising, scaling products, and leading teams to deliver production-grade AI solutions adopted globally.

See all products from TAI Labs

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.