Online Evals and Production Monitoring

Lightning Lessons

Practical skills & tools to accelerate your career

Online Evals and Production Monitoring

Hosted by Jason Liu and Ben Hylak

Wed, May 28, 2025

5:00 PM UTC (30 minutes)

Virtual (Zoom)

Free to join

Learn directly from undefined

By continuing, you agree to Maven's Terms and Privacy Policy.

114 students

By continuing, you agree to Maven's Terms and Privacy Policy.

Invite your network

Go deeper with a course

Systematically Improving RAG Applications

Jason Liu

View syllabus

Wed, May 28, 2025

5:00 PM UTC (30 minutes)

Virtual (Zoom)

Free to join

114 students

Invite your network

Go deeper with a course

Systematically Improving RAG Applications

Jason Liu

View syllabus

What you'll learn

AI Monitoring Fundamentals

Identify key metrics and set up effective monitoring systems to track AI product performance in real-time.

Issue Detection and Analysis

Learn techniques to pinpoint problems and segment performance by use case to quickly address critical issues.

Feedback-Driven Improvement

Apply user feedback patterns to prioritize product enhancements that directly increase retention and revenue

Why this topic matters

Mastering AI monitoring prevents costly failures that damage trust. This skill helps you catch issues before users do, drive data-informed improvements, and make strategic decisions that directly impact business outcomes. As AI deployment grows, monitoring expertise positions you to lead technical teams and protect company reputation.

You'll learn from

Jason Liu

Consultant at the intersection of Information Retrieval and AI

Jason has built search and recommendation systems for the past 6 years. He has consulted and advised a dozens startups in the last year to improve their RAG systems. He is the creator of the Instructor Python library.