Cohort 1

Follow a repeatable process to continually evaluate and improve your RAG application

Jason Liu

Systematically Improving RAG Applications

Stitch Fix

Google

Master RAG Development &amp; Transform Your AI Products Acquire the skills to confidently build, improve, and iterate on RAG applicationsBecome the RAG Product Leader Your Company Needs Companies are racing to integrate AI features that truly deliver value. To stay ahead, you must master the art of building Retrieval-Augmented Generation (RAG) products that continuously improve—no luck or guesswork required. 6 Weeks to Transform Your AI Product Sense This course removes the mystery and turns RAG from a risky experiment into a structured, data-driven practice. You'll learn how to pinpoint what's working, diagnose what's not, and steadily raise the bar on performance and user satisfaction. Course Format Our program follows a "flipped classroom" approach where you access all core lessons as pre-recorded videos to watch on your own schedule. This flexible format lets you learn at your own pace while still getting personalized support. Throughout each week, we offer multiple Office Hours sessions at various days and times. These optional sessions provide valuable opportunities to:- Ask questions about course material- Get direct feedback from Jason- Connect with fellow participants We recommend attending at least one Office Hour session per week. With multiple Office Hour sessions available, you'll have plenty of chances to get your specific questions answered directly by Jason in a small group environment. The curriculum is further enriched by special guest lectures from industry experts who bring additional perspectives and insights to complement the core material. What You'll Learn Master RAG Fundamentals — Build repeatable evaluation systems with synthetic data that drive measurable improvements. Decode User Behavior — Transform complex query patterns into actionable insights using powerful dashboards that prioritize high-impact changes. Conquer Multimodal Content — Extract precision metadata and create specialized indexes that handle diverse content formats without quality loss. Perfect Query Routing — Blend multiple search methods into a seamless system that intelligently directs queries for guaranteed results. Optimize Representations — Deploy domain-specific vectors and re-rankers that dramatically outperform generic embeddings with minimal data. Design Feedback-Rich UX — Create interfaces that eliminate perceived latency while collecting invaluable user signals for continuous improvement.﻿Why This MattersRAG has become a core differentiator for AI-driven products. The strategies you learn here will never go out of style. As tools and models evolve, you'll know how to integrate them, measure their impact, and refine your approach—over and over. What's Included - Proven evaluation frameworks that demystify "is it working?"- Techniques to handle more data types, faster and more accurately- Methods for continuous feedback collection and refinement- Insights that adapt as the AI landscape changes About Your Instructor Jason Liu is a seasoned AI consultant who has guided AI engineering at Stitch Fix and high-growth startups from Seed stage to public companies. He also advises many of today's generative AI startups. Jason specializes in transforming vague AI ambitions into concrete, measurable practices that deliver real business value. Who Should Apply - Product teams integrating AI features into real software products- AI engineers moving from prototype to production-grade performance- Anyone committed to developing true "AI product sense"—not just hype We're so confident in the course's value that if you don't see meaningful improvements in your product's performance within four weeks of starting, we'll refund you fully. Don't let uncertainty stall your AI progress. Join a small cohort of dedicated practitioners, learn the proven frameworks, and build the RAG systems your users will love. Getting started is easy. 1️⃣ Begin with my free 6-day RAG Playbook email course. You'll get bite-sized lessons delivered straight to your inbox, covering the fundamentals of RAG systems and practical tips for improvement at improvingrag.com 2️⃣ Join the 6-week cohort to master RAG basics and advanced implementation strategies. Perfect for those who deployed RAG systems and want to improve them and cover the last mile of RAG. 🔥 All alumni can stay up to date with our private community that includes personalized support, peer networking, and more! 📣 Live Office Hour sessions are at 9AM EST, 1PM EST and 2PM EST! You can pick whichever works best for you! All live sessions are recorded and the main content is delivered asynchronously so you can watch it on demand whenever you want. Interested in an enterprise deal so your whole team or company can take the course? Please reach out directly to support@jxnl.co!

Acquire the skills & confidently improving and iterate on RAG applications

A product leader, engineer, or data scientist looking to move beyond ad-hoc RAG prototypes into scalable, production-grade AI solutions.

A professional who understands LLM basics but wants a repeatable, data-driven methodology to improve retrieval relevance, latency, and user

Eager to create feedback loops that continuously refine and enhance the quality of RAG applications as models, data, and user needs evolve.

This Course Is For You If You Are

By the end of this course, participants will be able to

Implement the Data and Evals Flywheel approach to continuously develop and improve RAG applications—breaking free from guesswork and relying on measurable, iterative enhancements.

Adopt a Systematic, Data-First Methodology

Quickly assess your retrieval systems using precision and recall metrics, identify bottlenecks, and confidently validate changes without sinking into endless trial-and-error cycles.

Run Fast, Unit-Test-Like Evaluations

Generate and utilize synthetic data sets to speed up experimentation, enabling you to test new approaches, embeddings, and architectures before committing full resources.

Leverage Synthetic Data for Rapid Iteration

Apply fine-tuning strategies for embedding models to boost search relevance and explore hard negative examples to further sharpen retrieval performance.

Master Fine-Tuning & Hard Negative Mining

Use query classification and segmentation techniques to pinpoint exactly where your RAG system falls short—whether it’s due to limited inventory or insufficient capabilities.

Classify Queries & Identify Bottlenecks

Create tailored indices for documents, images, tables, SQL databases, and more. Learn when and how to fuse or layer these indices to handle complex retrieval tasks elegantly.

Design Specialized Indices for Multiple Modalities

Implement synthetic text chunk generation and strategic summarization methods to improve retrieval results, ensuring end-users get clear, concise, and contextually rich answers.

Enhance Retrieval with Summarization & Chunking

Develop systems that intelligently route queries to the right indices, tools, or pipelines. Blend and fuse indices effectively to handle nuanced, multi-step queries.

Implement Query Routing & Index Fusion

Evaluate the performance of your routing logic and each individual index separately. Gain the nuance to fine-tune global system performance and local retrieval accuracy in tandem.

Optimize Both Global & Local Performance

Design explicit and implicit feedback mechanisms—capturing user signals, automating re-labeling, and applying improvements in real-time to keep your RAG systems on an upward trajectory.

Integrate Feedback Loops for Continuous Improvement

Provide a brief description for each module as a way to sell the value of what students will be learning throughout the course.

First Module

Second Module

Third Module

Course syllabus

What people are saying

Meet your instructor

This course builds on live workshops and hands-on projects

Active hands-on learning

You’ll be interacting with other learners through breakout rooms and project teams

Interactive and project-based

Join a community of like-minded people who want to learn and grow alongside you

Course completed by

About this course

Get notified about the next cohort

Learn from live sessions