AI Eval Frameworks for UX and Product Teams
Melissa Wittmayer
Design Leader for agentic UX & evals
Design AI evals that measure what matters.
How do you know what “good” looks like for an AI agent?
Too many teams build an agent, ship it, and then sift through failures to figure out what went wrong. But how can you evaluate an experience if you haven't defined what success looks like in the first place?
Designers and product teams bring something essential to this work: an understanding of people, workflows, and human behavior. By applying that knowledge early, teams can define meaningful success criteria, anticipate failures, and build evaluations that measure what actually matters to users.
This hands-on cohort will teach you how to design AI evaluations from the ground up, whether you're starting from scratch or trying to make an existing evaluation practice more proactive. You'll map the full agent experience, identify the behaviors that matter, build a working evaluator, and turn evaluation results into product decisions.
Join the founding cohort: Save $200 with code FOUNDING200 at checkout.Be part of the first group working through this hands-on approach to designing AI evals.
What you’ll learn
Build a working evaluation practice you can apply to your own AI product. You'll leave with:
Map how workflows, agents, systems, and humans work together to accomplish a user goal.
Identify where the experience can break down, including failures that traditional task-based evaluations miss.
Run a collaborative exercise to identify the most consequential ways your agent could fail.
Turn those risks into a focused evaluation plan before investing in building evaluators.
Translate your desired user experience into observable behaviors.
Define a rubric and build an evaluator you can experiment with.
Learn how to focus your evaluations on the signals that matter most without creating unnecessary cost or complexity.
Practice diagnosing why an agent failed and deciding what to change.
Establish a shared approach your team can use to connect evaluation results to design, product, and engineering decisions.
Learn directly from Melissa
Melissa Wittmayer
Design Leader for agentic UX & evals, speaker, writer
Who this course is for
Designers or PMs trying to understand how to turn eval insights into solutions that improve the overall experience.
Product teams looking for ways to turn evals into a team practice.
Design leaders who want to support their teams by creating eval practices grounded by UX.
What's included
Live sessions
Learn directly from Melissa Wittmayer in a real-time, interactive format.
Lifetime access
Go back to course content and recordings whenever you need to.
Community of peers
Stay accountable and share insights with like-minded professionals.
Certificate of completion
Share your new skills with your employer or on LinkedIn.
Maven Guarantee
Your purchase is backed by the Maven Guarantee.
Course syllabus
4 live sessions • 12 lessons • 4 projects
Week 1
Nov
9
Session 1
Mon 11/98:00 PM—9:00 PM (UTC)Nov
13
Session 2
Fri 11/138:00 PM—9:00 PM (UTC)
Module 1: Building Agent Experiences
Module 2: Anticipating Failures
Week 2
Nov
16
Session 3
Mon 11/168:00 PM—9:00 PM (UTC)Nov
20
Session 4
Fri 11/208:00 PM—9:00 PM (UTC)
Module 3: Identifying Measurable Behaviors
Module 4: Evals as a Practice
Schedule
Live sessions
2 hrs / week
Mon, Nov 9
8:00 PM—9:00 PM (UTC)
Fri, Nov 13
8:00 PM—9:00 PM (UTC)
Mon, Nov 16
8:00 PM—9:00 PM (UTC)
Fri, Nov 20
8:00 PM—9:00 PM (UTC)
Projects
1 hr / week
Async content
2 hrs / week
Frequently asked questions
Maven for Teams
Reimbursement
Get your company to pay
Everything L&D needs: email template, receipts, and certificate of completion.
Get reimbursedTeam discount
Learn with your teammates
Save 20%+ when 2 or more teammates enroll in the same cohort.
Save 20%+ with a teamPrivate cohort
Run a cohort for your org
A dedicated cohort with a custom schedule and curriculum, tailored to your team.
Book a private cohort$750
USD