Synthetic RAG evaluation

Hosted by Alexey Grigorev and Doug Turnbull

168 students

What you'll learn

Generating Data for RAG Evaluation

Learn how to create synthetic datasets to test retrieval-augmented generation systems.

Basic Information Retrieval Metrics

Understand the core IR metrics used to measure retrieval effectiveness and how they apply to RAG.

Text vs. Vector Search — What's Better?

Explore the tradeoffs between keyword-based search and dense vector search for real-world applications.

Why this topic matters

Evaluating retrieval-augmented generation (RAG) is crucial for building reliable AI systems. By understanding how to generate evaluation data, apply IR metrics, and choose the right search method, you’ll be able to design more effective and trustworthy applications. These concepts give you the foundation to test and improve RAG pipelines — with practical insights you can apply immediately

You'll learn from

Alexey Grigorev

Principal Data Scientist | Book Author | Instructor to 100k+ Students World-Wide

Founder of DataTalks.Club · Creator of the Zoomcamp Series · ML Engineer & Author

Alexey Grigorev is the founder of DataTalks.Club and the creator of the popular Zoomcamp series. With 15 years of experience in software engineering and over 12 years in machine learning, he has built and deployed large-scale ML systems at companies like OLX Group and Simplaex.

An advocate for practical, hands-on education, Alexey has taught over 100,000 students, focusing on a code-first approach to help learners build real-world skills.

In the past, he was an active participant in data science competitions. A Kaggle Master, Alexey has achieved top rankings in several challenges, including 1st place in the NIPS'17 Criteo Challenge and 2nd place in the WSDM Cup 2017: Vandalism Detection.

He is also the author of several technical books, including the widely read Machine Learning Bookcamp.

Doug Turnbull

Principal ML Engineer in Search

Doug Turnbull has led search teams since 2013. He worked on AI in e-commerce going as far back an 2022 at Shopify. Delivered the largest experimental wins on Reddit search, and now advises AI and search teams as they take on modern search challenges. He teaches on Maven with his Cheat at Search and AI Powered Search courses.

Previously at