RAG Anti-patterns in the Wild, and How to Fix Them

Hosted by Jason Liu and Skylar Payne

Wed, Jun 11, 2025

5:00 PM UTC (1 hour)

Virtual (Zoom)

Free to join

107 students

Invite your network

Go deeper with a course

Systematically Improving RAG Applications
Jason Liu
View syllabus

What you'll learn

Diagnose Silent RAG Failures

Learn to identify subtle failure modes where retrieval systems pass tests but disappoint users in production.

Implement Robust Monitoring Strategies

Master practical techniques to detect hallucinations and relevance issues before they impact end users.

Apply Architectural Solutions

Gain concrete architectural patterns to transform struggling RAG systems into reliable production applications.

Why this topic matters

RAG systems often fail in production despite passing tests, silently frustrating users. By understanding common anti-patterns, you'll develop expertise to build reliable AI products that deliver on their promise. This knowledge transforms you from a RAG implementer to an architect of dependable systems that succeed in real-world environments.

You'll learn from

Jason Liu

Consultant at the intersection of Informational Retrieval and AI

Jason has built search and recommendation systems for the past 6 years. He has consulted and advised a dozens startups in the last year to improve their RAG systems. He is the creator of the Instructor Python library.

Skylar Payne

Founder of Wicked Data LLC

Skylar Payne specializes in rescuing AI systems that work in demos but fail in production. As founder of Wicked Data LLC, he diagnoses and fixes broken retrieval systems for startups and enterprises alike. Previously, as VP of Engineering & Data Science at HealthRhythms, he built monitoring systems that caught silent ML failures before they reached users. At LinkedIn, he led teams that increased experimentation velocity 3x while moving core metrics by double digits through improved recommendation systems. At Google, he developed testing infrastructure for large-scale distributed systems. Skylar has personally debugged retrieval systems across every scale—from early prototypes to platforms serving hundreds of millions—and brings battle-tested patterns that bridge the gap between ML experiments and reliable products.

Previously at

Stitch Fix
Meta
University of Waterloo
New York University

Learn directly from Jason Liu and Skylar Payne

By continuing, you agree to Maven's Terms and Privacy Policy.

© 2025 Maven Learning, Inc.