Systematically Improving RAG Applications

New
·

6 Weeks

·

Cohort-based Course

Follow a repeatable process to continually evaluate and improve your RAG application

Last chance to apply

4

days

6

hours

23

mins

Instructors' Experience

Stitch Fix
Google
Straive
Meta
Weights & Biases

Course overview

Learn and implement the playbook for success with RAG

You're working on a new RAG application in a chaotic environment. Competing priorities and limited resources make progress challenging, with each day feeling like a roll of the dice. Success hinges on finding "the right path" - a sequence of actions for maximum growth in minimal time.


Beyond technical skills, effective AI development requires strategic decision-making and resource allocation.


Key abilities include:


1. Identifying high-impact tasks

2. Prioritizing effectively

3. Making informed tradeoffs

4. Choosing relevant metrics

5. Focusing on what matters most


These skills, often overlooked, are crucial for navigating the complexities of AI development and maximizing progress with limited time and resources. Mastering them can transform the chaotic development process into a more directed and efficient journey.


After taking this course, you'll become a leader who can:


- Prioritize effectively in your search pipeline

- Distill problems into key retrieval and generation metrics

- Establish an Evaluation Pipeline for RAG applications

- Preemptively identify and address major issues

- Gain team buy-in through data-supported strategies


This comprehensive playbook will enable you to deliver consultant-level value, leading your team to results through structured experimentation. We guarantee meaningful improvements in processes in 5 weeks or we'll personally refund you and do a free consultation to help figure out your needs.


Our consulting has achieved:


- 80%+ satisfaction in previously unsuccessful query segments

- 4x increase in feedback volume and 10% boost in CSAT

- Finetune embedding models that outperform closed models by 20-30%

- 2-3x increase in weekly experiments using synthetic data benchmarks

- Real-time monitoring systems revealing new product positioning insights


Don't do it alone - be part of a small cohort of other teams shipping real applications.


To ensure everyone can be successful, the course requires a short application. You pay only after your application is approved.


Get these free bonuses (over $1500 in value):

$500 Cohere credits (Jason uses Cohere rerankers in every single RAG product he's build or adviced)

$200 LanceDB credits and free access to Lance Cloud

$500 in Modal Labs credits (useful for experimenting with embedding fine-tuning)

6 months free Notion AI Plus (get experience with more RAG products)

3 months Braintrust access ($250 value)

This Course Is For You If You Are

01

An Engineering or Product leader looking to improve an existing RAG system MVP

02

Solving problems like poor retrieval, unreliable outputs or unhappy customers with your existing application

03

Ready to lead your team in building a data flywheel so you can leverage feedback

What you’ll get out of this course

Join a community of operators that are also shipping production applications
Identify and test improvements over time by developing a experimentation platform


Specialize your search backends to serve the questions that matter to your business


Identify differences in query satisfaction across segments, so you focus on key improvement areas


Apply techniques including rerankers and ColBERT to improve search quality while managing system latency


Learn to handle both data idiosyncrasies and organizational challenges from guest instructors who have deployed successful RAG applications
Experiment with credits, tools, and talks from others building in the space

  • Talks from Cohere, Vespa, LanceDB, Zapier, Braintrust, Answer.AI on how to improve systems, not just vendor talks.
  • Credits from Modal to explore fine tuning embedding models and rerankers
  • Talk from New Computer on how they improved memory
  • and more!

This course includes

16 interactive live sessions

Lifetime access to course materials

32 in-depth lessons

Direct access to instructor

Projects to apply learnings

Guided feedback & reflection

Private community of peers

Course certificate upon completion

Maven Satisfaction Guarantee

This course is backed by Maven’s guarantee. You can receive a full refund within 14 days after the course ends, provided you meet the completion criteria in our refund policy.

Course syllabus

Expand all modules
  • Week 1

    Jul 30—Aug 4

    Week dates are set to instructor's time zone

    Events

    • Jul

      30

      Intro to the Playbook + RAG Evaluation

      Tue, Jul 30, 5:00 PM - 6:00 PM UTC

    • Jul

      31

      Breakout + Office Hours

      Wed, Jul 31, 5:00 PM - 6:00 PM UTC

    • Aug

      1

      Guest Speaker session

      Thu, Aug 1, 5:00 PM - 6:00 PM UTC

    Modules

    • Intro To The Playbook

    • RAG Evaluation

    • Synthetic Data

  • Week 2

    Aug 5—Aug 11

    Week dates are set to instructor's time zone

    Events

    • Aug

      6

      Identifying Areas of Improvement

      Tue, Aug 6, 5:00 PM - 6:00 PM UTC

    • Aug

      7

      Breakout + Office Hours

      Wed, Aug 7, 5:00 PM - 6:00 PM UTC

    • Aug

      8

      Guest speaker session

      Thu, Aug 8, 5:00 PM - 6:00 PM UTC

    Modules

    • Identifying Areas of Improvement

    • Production Monitoring and Analysis

  • Week 3

    Aug 12—Aug 18

    Week dates are set to instructor's time zone

    Events

    • Aug

      13

      IR Keys + Non-Text Data

      Tue, Aug 13, 5:00 PM - 6:00 PM UTC

    • Aug

      14

      Breakout + Office Hours

      Wed, Aug 14, 5:00 PM - 6:00 PM UTC

    • Aug

      15

      Guest Speaker Session

      Thu, Aug 15, 5:00 PM - 6:00 PM UTC

    Modules

    • Improving Retrieval

    • Tables and Non-Text Data

  • Week 4

    Aug 19—Aug 25

    Week dates are set to instructor's time zone

    Events

    • Aug

      20

      Routing Queries

      Tue, Aug 20, 5:00 PM - 6:00 PM UTC

    • Aug

      21

      Breakout + Office Hours

      Wed, Aug 21, 5:00 PM - 6:00 PM UTC

    • Aug

      22

      Guest Speaker Session

      Thu, Aug 22, 5:00 PM - 6:00 PM UTC

    Modules

    • Routing Queries

  • Week 5

    Aug 26—Sep 1

    Week dates are set to instructor's time zone

    Events

    • Aug

      27

      Representations

      Tue, Aug 27, 5:00 PM - 6:00 PM UTC

    • Aug

      28

      Breakout + Office Hours

      Wed, Aug 28, 5:00 PM - 6:00 PM UTC

    • Aug

      29

      Guest Speaker Session

      Thu, Aug 29, 5:00 PM - 6:00 PM UTC

    Modules

    • Representations

    • Synthetic Text Chunks

  • Week 6

    Sep 2—Sep 6

    Week dates are set to instructor's time zone

    Events

    • Sep

      4

      Breakout + Office Hours

      Wed, Sep 4, 5:00 PM - 6:00 PM UTC

  • Post-Course

    Modules

    • Product Design

    • Rejecting work

Meet your instructor

Jason Liu

Jason Liu

Jason has built search and recommendation systems for the past 6 years. He has consulted and advised a dozens startups in the last year to improve their RAG systems. He is the creator of the Instructor Python library.

Dan Becker

Dan Becker

Dan has worked in AI since 2011, when he finished 2nd (out of 1350+ teams) in a Kaggle competition with a $500k prize. He contributed code to TensorFlow as a data scientist at Google and he has taught online deep learning courses to over 250k people. Dan has advised AI projects for 6 companies in the Fortune 100.

A pattern of wavy dots
Join an upcoming cohort

Systematically Improving RAG Applications

Cohort 1

$1,650

Dates

July 30—Sep 6, 2024

Application Deadline

July 30, 2024

Don't miss out! Applications close in 5 days

|

Bulk purchases

Course Schedule Each Week

  • Tuesday: Workshops

    1:00 - 2:00PM ET

    Workshops covering each step of the playbook and helping you build process improvements in your RAG application

  • Wednesday: Office Hours + Breakout Sessions

    1:00 - 2:00PM ET

    The first half hour will be interactive breakout sessions, and the closing half-hour each week is Q&A

  • Thursday: Guest Speakers

    1:00 - 2:00PM ET

    Guest instructors covering key topics in both innovative theory and practical applications in RAG system development.

Learning is better with cohorts

Learning is better with cohorts

Active hands-on learning

This course builds on live workshops and hands-on projects

Interactive and project-based

You’ll be interacting with other learners through breakout rooms and project teams

Learn with a cohort of peers

Join a community of like-minded people who want to learn and grow alongside you

Frequently Asked Questions

What happens if I can’t make a live session?
I work full-time, what is the expected time commitment?
What’s the refund policy?

Stay in the loop

Sign up to be the first to know about course updates.

A pattern of wavy dots
Join an upcoming cohort

Systematically Improving RAG Applications

Cohort 1

$1,650

Dates

July 30—Sep 6, 2024

Application Deadline

July 30, 2024

Don't miss out! Applications close in 5 days

|

Bulk purchases

$1,650

6 Weeks