Interleaving for better RAG evaluation

Featured

Interleaving for better RAG evaluation

Hosted by Max Irwin, Doug Turnbull, and Trey Grainger

Tue, Nov 18, 2025

6:00 PM UTC (1 hour)

Virtual (Zoom)

Free to join

By continuing, you agree to Maven's Terms and Privacy Policy.

76 students

By continuing, you agree to Maven's Terms and Privacy Policy.

Invite your network

Go deeper with a course

AI-Powered Search: Modern Retrieval for Humans & Agents

Doug Turnbull and Trey Grainger

View syllabus

Tue, Nov 18, 2025

6:00 PM UTC (1 hour)

Virtual (Zoom)

Free to join

76 students

Invite your network

Go deeper with a course

AI-Powered Search: Modern Retrieval for Humans & Agents

Doug Turnbull and Trey Grainger

View syllabus

What you'll learn

Why summaries silently kill your retriever metrics

Why an LLM summary above search results reduces clicks, starves your A/B tests, and eliminates your ability to evaluate

Team-draft interleaving for RAG (not just for humans)

How team-draft interleaving blends results from two or more engines into a single ranked list. How to apply it to RAG

Using LLM citations as implicit relevance judgments

Treating summary citations to credit each retriever: a direct, comparable measure of the LLM's preference

Compare multiple retrievers and multiple LLMs at once

Three example search engines + Four LLM models to answer which retriever works best for my RAG stack?

Why this topic matters

Most teams still tune RAG retrievers like classic search—optimize NDCG, pick a config, move on. But summaries change everything: users read, don’t click, and metrics go blind. Interleaving for RAG blends multiple retrievers, lets the LLM summarize with citations, and shows which retriever actually powered the answer—giving real feedback in a post-click world.

You'll learn from

Max Irwin

Founder/CEO, Max.io

Max has 25 years of experience working in search as a hands-on developer, strategist, and leader. Prior to forming MAX.IO, I was managing consultant at OpenSource Connections, where I worked with dozens of companies to improve search quality, trained hundreds of developers in search and machine learning, and was a frequent speaker and blogger. I'm also a contributing author of "AI Powered Search".

Doug Turnbull

Search consultant

Doug Turnbull has led search teams since 2013. He worked on AI in e-commerce going as far back an 2022 at Shopify. Delivered the largest experimental wins on Reddit search, and now advises AI and search teams as they take on modern search challenges. He is co-author of the book AI-Powered Search and teaches on Maven with his Cheat at Search and AI-Powered Search courses.

Trey Grainger

Founder & CEO SearchKernel

Trey is lead author of the book AI-Powered Search and is the founder of Searchkernel, a software company building the next generation of AI-powered search. He is an advisor to several startups and adjunct professor of computer science at Furman University. He previously served as CTO of Presearch, a decentralized web search engine, and as chief algorithms officer and SVP of engineering at Lucidworks, an search company whose search technology powers hundreds of the world’s leading organizations.

Previously at