Free Lesson

Evaluate AI Agents with Simulated Users

Hosted by Stella Liu

Fri, Aug 7, 2026

7:00 PM UTC (30 minutes)

Virtual (Zoom)

Free to join

Invite your network

Go deeper with a course

AI Evals and Analytics Playbook
Stella Liu and Amy Chen
View syllabus

What you'll learn

Understand how user simulation works

See how an LLM-based user simulator interacts with an agent

Create real-world-like evaluation scenarios

Define user context, data states, policies, and measurable success criteria.

Build and run a custom simulation with τ²-bench

Follow a live end-to-end implementation of a student financial-aid domain.

Why this topic matters

Agents rarely succeed or fail because of a single response. They must manage evolving conversations, request missing information, call the correct tools, follow policies, and complete the user’s task. User simulation gives us a repeatable way to test these interactions before they reach real users. In this lightning lesson, let's walk through how to use τ²-bench to build user simulation.

You'll learn from

Stella Liu

Head of Applied Scientist, AI Evals

Stella Liu is an AI Evaluation practitioner and researcher, specializing in frameworks for large language models and AI-powered products.


Since 2023, she has led real-world AI evaluation projects in EdTech, where she established the first AI product evaluation framework for Higher Education and continues to advance research on the safe and responsible use of AI. Her work combines academic rigor with hands-on product experience, bringing proven evaluation methods into both enterprise and educational contexts.


Earlier in her career, Stella worked at Shopify and Carvana, where she built large-scale data-driven automation systems that powered product innovation and operational efficiency at scale.


Stella also writes an AI Evals newsletter on Substack: https://datasciencexai.substack.com/


Follow Stella on LinkedIn

Previously at

Shopify.com
Carvana
Harvard University
UT Austin
See all products from AI Evals and Analytics

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.