Dorra Mlouhi
Ahmed Wafaey
Rituals.pm
Free Lesson

How AI Work as a PM (part 4):Eval Hands-on Case Study

60 min
Nov 29, 2026 12:30 PM

By continuing, you agree to Maven's Terms and Privacy Policy.

What you'll learn

Turn “good” into test cases

Write the criteria and a golden dataset for a real AI feature, including edge cases and adversarial inputs.

Set up the right graders.

Use code checks for rules and an LLM judge for nuance, then check the judge against human scores.

Run the eval loop

Score the results, spot failure patterns, and decide whether to fix the prompt, the tools or the guardrails.

Why this topic matters

Everyone says “you need evals.” Few teams actually have them. In this hands-on session, we take one real AI feature and build its eval suite together, live: define what good looks like, write a golden dataset, set up code checks and an LLM judge, score the results, read the failures and improve. You’ll leave with a reusable eval template and the confidence to run evals on your own product. Bring a use case, or use ours.

You'll learn from

Dorra Mlouhi

Dorra Mlouhi

Product Leader and AI Builder

Ahmed Wafaey

Ahmed Wafaey

Product Leader and AI Practitioner

Rituals.pm

Rituals.pm

Product Leaders and AI Practitioners (AWS, Delivery Hero, Uber...)

Amazon Web Services
Uber
Delivery Hero
Tesco
Trivago
See all products from Rituals.pm
Get free access