Cheat at Search with Agents

Nov 2025

Cohort 1

October 2025

Feb 2026

July 2025

June 2026

Rethink search to put tireless agents at work for human users

http://softwaredoug.com

Doug Turnbull

Snuggie and Search Enthusiast

Product Manager

full-stack developer

AI Engineer & Psychologist

Software Engineer

Consulting Architect

Student

Trainer, Lecturer, Author

Lead Engineer

Computer Engineer

Humans increasingly search via chat, not search bars. Agentic search goes beyond RAG to incorporate agentic reasoning into the search process. Agents iterate, refine, and learn to use search tools in away monolithic RAG fails to.Come to this class to:<ul><li>Leverage agentic reasoning to go beyond RAG - Traditional RAG can't reason. An agent learns about retrieval systems and tailors its searches for more accurate results. Go beyond naive RAG towards agentic search</li><li>Craft effective retrieval - Focus an agent's reasoning on the user's task, not "search", with task-targeted search tools. Tools built for agents; not for humans.</li><li>Go beyond LLM as a judge - Integrate human and clickstream feedback to guide the agent's judgment towards what users want</li><li>Improve keyword search - Take lessons learned through agentic search back to traditional "search bar" search, simplifying query and content understanding</li></ul>Come learn alongside search industry expert <a target="_blank" rel="noopener noreferrer nofollow" href="http://softwaredoug.com">Doug Turnbull</a> as we unpack how agents will change the search industry.This class is a fun and experimental course, focused on play and experimentation. It's part of my quest to find new, practical frontiers in Information Retrieval. I hope you can join me!

Agents are the new search engine. Go beyond RAG and keywords.

What makes a great search tool to an agent?

Use LLMs to help agents structure, organize, and find content

Agentic results without the cost

Orienting agentic search towards the human, not the machi

We will build on top of existing, core search knowledge

Lexical Search basics (do you know what BM25 is)

We will use common search datasets that use metrics like NDCG, etc to measure search

Basic search evaluation (judgments + metrics)

Vector search will be one tool in our toolkit we'll apply to the agentic tools that we build

Vector search basics (do you know what an embedding is?)

Search technologists that want to learn how to ensure RAG and agentic search win

Startups eager to get a fast start with a modern approach to search, sidestepping old, outdated practices

Data scientists that want to consider how agentic / LLM / RAG based search can be oriented towards user feedback and business outcomes

Adopting Agentic E-Commerce Search

John Berryman

Arcturus Labs

Search + Snuggie Enthusiast

Learn how RAG and agents are really simple pipelines and loops around existing search APIs—not magic systems.

Demystifying RAG and Agentic AI

See how to introduce AI suggestions, query interpretation, and assistants while keeping your current search backend.

Adding AI Without Rewriting Your Stack

Offer a concrete and concise explanation of how you will
help students understand and apply this lesson.


Practical AI Adoption for E-Commerce Teams

Tuning HNSW parameters for filtered search

Radu Gheorghe

Software Engineer, Vespa.ai

Retrieval + Snuggie Enthusiast

We'll start with a short intro on how HNSW works and why filters are expensive and hurt recall.

Why filters make HNSW vector search slow

Pros and cons of post filters, ACORN-1, brute force kNN, overfetching, and adaptive beam search.

Parameters to improve performance/recall

How HNSWTuner+VespaNNParameterOptimizer allows you to change these knobs and see the impact of latency and recall

Tools to tune vector search filter search

Build a vector database from scratch

Search Consultant and Coach

Founding member Github Copilot

Learn the most common graph-based vector search algorithm used in Elasticsearch, Weaviate, QDrant, Pinecone, etc

Learn how embedding similarity search works

As you watch an expert live-code, see where the algorithm falls apart as a real-life expert makes a mistake

See where vector search goes wrong

Dense vectors provide specific constraints on retrieval solutions - learn what these are, how they can go wrong

Deepen your vector search knowledge

Could your search be better without vectors? BM25 + friends

Learn the baseline that rules them all (BM25) and how it works

Basics of Information Retrieval

How the simple decision of how to break up strings can make or break search

The importance of tokenization

How to achieve semantic search without vector search - and why you might prefer a lexical semantic approach

Semantic search without the vectors

Cheat at Search Essentials: Evaluation, NDCG, and pals

Ex-Reddit, Ex-Shopify. Author: AI Powered Search+Relevant Search

How the best search teams evaluate and measure search relevance

Basics of search relevance evaluation

When to ignore traditional search metrics and just trust your A/B tests. How to interpret offline metrics.

Where practice diverges from theory

Do you use clicks? Human labels? LLM as a judge? What are the pros/cons of these approaches?

Pros / cons of different types of labeled relevance data

Cheat at Search Essentials: Vectors and Embeddings

Ex-Reddit, Ex-Shopify Search

What is a 'vector embedding' and why do they capture meaning? And how can that be used to build a search system?

Learn how embeddings implement 'semantic search'

Why do we need a vector database? And why are they different from traditional search engines? Should they be different?

Tackle the practical realities vector retrieval

How are vector + lexical techniques combined?

CTO's Playbook for Faster Agentic RAG

Nirant Kasiwal

Creator FastEmbed

Led Search at Reddit, Shopify, Wikipedia

know exactly when to swap OpenAI for FastEmbed and improve both recall and latency.

Latency vs Recall Cheat-Sheet

live-coded: BM25 + vector ➜ tool-calling agent

Hybrid-to-Agentic Upgrade Path

7-point checklist that spots 90 % of real-world retrieval failures before users do.

Stealable RAG Audit Framework

Cheat at Search Essentials: BM25 + Lexical

Principal AI Engineer in Search

Cheat at semantic search without the vectors

Building Deep Research Systems

Andrei Lopatenko

Search at Zillow, Google, eBay, Apple

Agentic Search Consultant

How retrieval augmentation and semantic grounding expand LLM capabilities for knowledge discovery.

Pushing beyond the LLMs knowledge

The design principles behind multi-hop reasoning and deep exploration architectures.

Deep Research Design Principles

Evaluation methods and frameworks shaping next-generation research and information systems.

The next generation of search eval

Navigating your tech career in the age of layoffs and AI

Team Lead at Shopify, Reddit, and AI Startups

Brian Pedersen

Principal, "The Search Bar"

What tools and skills should you acquire to thrive in with ubiquitious AI and looming layoffs?

How can I future-proof my career?

How to match up what makes you happy with the needs of the job market

What does fulfillment look like?

What skills do hiring managers focus on increasingly? What can help you standout on your team, or in the next interview?

Learn directly from Cheat at Search with Agents