Systematically Improving RAG Applications

4.7 (29)

·

4 Weeks

·

Cohort-based Course

Follow a repeatable process to continually evaluate and improve your RAG application

Instructor Clients

Stitch Fix
Meta
Google

Course overview

Acquire the skills & confidently improving and iterate on RAG applications

Master RAG Development & Transform Your AI Products


Acquire the skills to confidently build, improve, and iterate on RAG applications

Become the RAG Product Leader Your Company Needs


Companies are racing to integrate AI features that truly deliver value. To stay ahead, you must master the art of building Retrieval-Augmented Generation (RAG) products that continuously improve—no luck or guesswork required.


4 Weeks to Transform Your AI Product Sense


This course removes the mystery and turns RAG from a risky experiment into a structured, data-driven practice. You'll learn how to pinpoint what's working, diagnose what's not, and steadily raise the bar on performance and user satisfaction.


Course Format


Our program follows a "flipped classroom" approach where you access all core lessons as pre-recorded videos to watch on your own schedule. This flexible format lets you learn at your own pace while still getting personalized support.


Throughout each week, we offer multiple Office Hours sessions at various days and times. These optional sessions provide valuable opportunities to:

- Ask questions about course material

- Get direct feedback from Jason

- Connect with fellow participants


We recommend attending at least one Office Hour session per week. With multiple Office Hour sessions available, you'll have plenty of chances to get your specific questions answered directly by Jason in a small group environment.


The curriculum is further enriched by special guest lectures from industry experts who bring additional perspectives and insights to complement the core material.


What You'll Learn


Master RAG Fundamentals — Build repeatable evaluation systems with synthetic data that drive measurable improvements.


Decode User Behavior — Transform complex query patterns into actionable insights using powerful dashboards that prioritize high-impact changes.


Conquer Multimodal Content — Extract precision metadata and create specialized indexes that handle diverse content formats without quality loss.


Perfect Query Routing — Blend multiple search methods into a seamless system that intelligently directs queries for guaranteed results.


Optimize Representations — Deploy domain-specific vectors and re-rankers that dramatically outperform generic embeddings with minimal data.


Design Feedback-Rich UX — Create interfaces that eliminate perceived latency while collecting invaluable user signals for continuous improvement.



Why This Matters

RAG has become a core differentiator for AI-driven products. The strategies you learn here will never go out of style. As tools and models evolve, you'll know how to integrate them, measure their impact, and refine your approach—over and over.


What's Included


- Proven evaluation frameworks that demystify "is it working?"

- Techniques to handle more data types, faster and more accurately

- Methods for continuous feedback collection and refinement

- Insights that adapt as the AI landscape changes


About Your Instructor


Jason Liu is a seasoned AI consultant who has guided AI engineering at Stitch Fix and high-growth startups from Seed stage to public companies.


He also advises many of today's generative AI startups. Jason specializes in transforming vague AI ambitions into concrete, measurable practices that deliver real business value.


Who Should Apply


- Product teams integrating AI features into real software products

- AI engineers moving from prototype to production-grade performance

- Anyone committed to developing true "AI product sense"—not just hype


We're so confident in the course's value that if you don't see meaningful improvements in your product's performance within four weeks of starting, we'll refund you fully.

Don't let uncertainty stall your AI progress.


Join a small cohort of dedicated practitioners, learn the proven frameworks, and build the RAG systems your users will love.




Getting started is easy.


1️⃣ Begin with my free 6-day RAG Playbook email course. You'll get bite-sized lessons delivered straight to your inbox, covering the fundamentals of RAG systems and practical tips for improvement at improvingrag.com


2️⃣ Join the 4-week cohort to master RAG basics and advanced implementation strategies. Perfect for those who deployed RAG systems and want to improve them and cover the last mile of RAG.


🔥 All alumni can stay up to date with our private community that includes personalized support, peer networking, and more!


📣 Live Office Hour sessions are at 9AM EST, 1PM EST and 2PM EST! You can pick whichever works best for you! All live sessions are recorded and the main content is delivered asynchronously so you can watch it on demand whenever you want.


Interested in an enterprise deal so your whole team or company can take the course? Please reach out directly to support@jxnl.co!

This Course Is For You If You Are

01

A product leader, engineer, or data scientist looking to move beyond ad-hoc RAG prototypes into scalable, production-grade AI solutions.

02

A professional who understands LLM basics but wants a repeatable, data-driven methodology to improve retrieval relevance, latency, and user

03

Eager to create feedback loops that continuously refine and enhance the quality of RAG applications as models, data, and user needs evolve.

By the end of this course, participants will be able to

Adopt a Systematic, Data-First Methodology

Implement the Data and Evals Flywheel approach to continuously develop and improve RAG applications—breaking free from guesswork and relying on measurable, iterative enhancements.

Run Fast, Unit-Test-Like Evaluations

Quickly assess your retrieval systems using precision and recall metrics, identify bottlenecks, and confidently validate changes without sinking into endless trial-and-error cycles.

Leverage Synthetic Data for Rapid Iteration

Generate and utilize synthetic data sets to speed up experimentation, enabling you to test new approaches, embeddings, and architectures before committing full resources.

Master Fine-Tuning & Hard Negative Mining

Apply fine-tuning strategies for embedding models to boost search relevance and explore hard negative examples to further sharpen retrieval performance.

Classify Queries & Identify Bottlenecks

Use query classification and segmentation techniques to pinpoint exactly where your RAG system falls short—whether it’s due to limited inventory or insufficient capabilities.

Design Specialized Indices for Multiple Modalities

Create tailored indices for documents, images, tables, SQL databases, and more. Learn when and how to fuse or layer these indices to handle complex retrieval tasks elegantly.

Enhance Retrieval with Summarization & Chunking

Implement synthetic text chunk generation and strategic summarization methods to improve retrieval results, ensuring end-users get clear, concise, and contextually rich answers.

Implement Query Routing & Index Fusion

Develop systems that intelligently route queries to the right indices, tools, or pipelines. Blend and fuse indices effectively to handle nuanced, multi-step queries.

Optimize Both Global & Local Performance

Evaluate the performance of your routing logic and each individual index separately. Gain the nuance to fine-tune global system performance and local retrieval accuracy in tandem.


Integrate Feedback Loops for Continuous Improvement

Design explicit and implicit feedback mechanisms—capturing user signals, automating re-labeling, and applying improvements in real-time to keep your RAG systems on an upward trajectory.

This course includes

18 interactive live sessions

Lifetime access to course materials

25 in-depth lessons

Direct access to instructor

Projects to apply learnings

Guided feedback & reflection

Private community of peers

Course certificate upon completion

Maven Satisfaction Guarantee

This course is backed by Maven’s guarantee. You can receive a full refund within 14 days after the course ends, provided you meet the completion criteria in our refund policy.

Course syllabus

Week 1

May 20—May 25

    Week 1

    • May

      20

      Introductions

      Tue 5/203:00 PM—4:00 PM (UTC)
    • May

      21

      Guest Lecture [Coming Soon]

      Wed 5/215:00 PM—6:00 PM (UTC)

    Office Hours

    • May

      20

      Office Hour

      Tue 5/201:00 PM—2:00 PM (UTC)
      Optional
    • May

      20

      Office Hour

      Tue 5/205:00 PM—6:00 PM (UTC)
      Optional
    • May

      22

      Office Hour

      Thu 5/226:00 PM—7:00 PM (UTC)
      Optional

Week 2

May 26—Jun 1

    Week 2

    • May

      28

      Guest Lecture [Coming Soon]

      Wed 5/285:00 PM—6:00 PM (UTC)

    Office Hours

    • May

      27

      Office Hour

      Tue 5/271:00 PM—2:00 PM (UTC)
      Optional
    • May

      27

      Office Hour

      Tue 5/275:00 PM—6:00 PM (UTC)
      Optional
    • May

      29

      Office Hour

      Thu 5/296:00 PM—7:00 PM (UTC)
      Optional

Week 3

Jun 2—Jun 8

    Week 3

    • Jun

      4

      Guest Lecture [Coming Soon]

      Wed 6/45:00 PM—6:00 PM (UTC)

    Office Hours

    • Jun

      3

      Office Hour

      Tue 6/31:00 PM—2:00 PM (UTC)
      Optional
    • Jun

      3

      Office Hour

      Tue 6/35:00 PM—6:00 PM (UTC)
      Optional
    • Jun

      5

      Office Hour

      Thu 6/56:00 PM—7:00 PM (UTC)
      Optional

Week 4

Jun 9—Jun 12

    Week 4

    • Jun

      11

      Guest Lecture [Coming Soon]

      Wed 6/115:00 PM—6:00 PM (UTC)
    • Jun

      12

      Conclusions

      Thu 6/128:00 PM—9:00 PM (UTC)

    Office Hours

    • Jun

      10

      Office Hour

      Tue 6/101:00 PM—2:00 PM (UTC)
      Optional
    • Jun

      10

      Office Hour

      Tue 6/105:00 PM—6:00 PM (UTC)
      Optional
    • Jun

      12

      Office Hour

      Thu 6/126:00 PM—7:00 PM (UTC)
      Optional

Bonus

4.7 (29 ratings)

What students are saying

What people are saying

        As an Applied AI Engineer at Anthropic, I was familiar with all of the standard retrieval methods and RAG papers going into the course, but Jason's frameworks helped me to operationalize what I'd learned and it's had an incredibly positive impact in my work with customers.
Sam Flamini

Sam Flamini

Solutions Engineer at Anthropic
        Evals really moving us forward again and past the "vibe check" plateau. First iteration alone has highlighted multiple non-obvious failure modes of the system. In combination with customer feedback / bug reports / traces. So satisfying to have good visibility again into where we can get some easy wins.
Nico Neven

Nico Neven

CTO at Vantager

Meet your instructor

Jason Liu

Jason Liu

Jason has built search and recommendation systems for the past 6 years. He has consulted and advised a dozens startups in the last year to improve their RAG systems. He is the creator of the Instructor Python library.

A pattern of wavy dots

Join an upcoming cohort

Systematically Improving RAG Applications

Cohort 3

$1,800

Dates

May 20—June 12, 2025

Payment Deadline

May 21, 2025
Get reimbursed

Learning is better with cohorts

Learning is better with cohorts

Active hands-on learning

This course builds on live workshops and hands-on projects

Interactive and project-based

You’ll be interacting with other learners through breakout rooms and project teams

Learn with a cohort of peers

Join a community of like-minded people who want to learn and grow alongside you

Frequently Asked Questions

Stay in the loop

Sign up to be the first to know about course updates.

A pattern of wavy dots

Join an upcoming cohort

Systematically Improving RAG Applications

Cohort 3

$1,800

Dates

May 20—June 12, 2025

Payment Deadline

May 21, 2025
Get reimbursed

$1,800

4.7 (29)

·

4 Weeks