Free Lesson

Evaluate AI Agents with Simulated Users

30 min
Aug 7, 2026 3:00 PM
Virtual (Zoom)

In this video

What you'll learn

Understand how user simulation works

See how an LLM-based user simulator interacts with an agent

Create real-world-like evaluation scenarios

Define user context, data states, policies, and measurable success criteria.

Build and run a custom simulation with τ²-bench

Follow a live end-to-end implementation of a student financial-aid domain.

Why this topic matters

Agents rarely succeed or fail because of a single response. They must manage evolving conversations, request missing information, call the correct tools, follow policies, and complete the user’s task. User simulation gives us a repeatable way to test these interactions before they reach real users. In this lightning lesson, let's walk through how to use τ²-bench to build user simulation.

You'll learn from

Stella Liu

Stella Liu

Head of Applied Scientist, AI Evals

Previously at

Shopify.com
Carvana
Harvard University
UT Austin
See all products from AI Evals and Analytics