Cheat at Search with Agents

Nov 2025

Cohort 1

October 2025

Feb 2026

July 2025

June 2026

Rethink search to put tireless agents at work for human users

http://softwaredoug.com

Doug Turnbull

Led Search Reddit + Shopify. Wrote Relevant Search + AI Powered Search

Product Manager

Principal Engineer

Indie Consultant

Software Engineer

Principal recommender engineer 

Search Consultant

Generative AI Engineer

Software Engineer Search

Engineering Manager

Humans increasingly search via chat, not search bars. Agentic search goes beyond RAG to incorporate agentic reasoning into the search process. Agents iterate, refine, and learn to use search tools in away monolithic RAG fails to.Come to this class to:<ul><li>Leverage agentic reasoning to go beyond RAG - Traditional RAG can't reason. An agent learns about retrieval systems and tailors its searches for more accurate results. Go beyond naive RAG towards agentic search</li><li>Craft effective retrieval - Focus an agent's reasoning on the user's task, not "search", with task-targeted search tools. Tools built for agents; not for humans.</li><li>Go beyond LLM as a judge - Integrate human and clickstream feedback to guide the agent's judgment towards what users want</li><li>Improve keyword search - Take lessons learned through agentic search back to traditional "search bar" search, simplifying query and content understanding</li></ul>Come learn alongside search industry expert <a target="_blank" rel="noopener noreferrer nofollow" href="http://softwaredoug.com">Doug Turnbull</a> as we unpack how agents will change the search industry.This class is a fun and experimental course, focused on play and experimentation. It's part of my quest to find new, practical frontiers in Information Retrieval. I hope you can join me!

Agents are the new search engine. Go beyond RAG and keywords.

What makes a great search tool to an agent?

Use LLMs to help agents structure, organize, and find content

Agentic results without the cost

Orienting agentic search towards the human, not the machi

We will build on top of existing, core search knowledge

Lexical Search basics (do you know what BM25 is)

We will use common search datasets that use metrics like NDCG, etc to measure search

Basic search evaluation (judgments + metrics)

Vector search will be one tool in our toolkit we'll apply to the agentic tools that we build

Vector search basics (do you know what an embedding is?)

Search technologists that want to learn how to ensure RAG and agentic search win

Startups eager to get a fast start with a modern approach to search, sidestepping old, outdated practices

Data scientists that want to consider how agentic / LLM / RAG based search can be oriented towards user feedback and business outcomes

Avoiding expensive AI surprises with LiteLLM

Ewan Nisbet

Technologist

Search Practitioner

use local proxies like LiteLLM instead of relying on opaque hyperscaler abstractions to manage LLM use

Put real cost and usage boundaries around LLM experiments

CLI tools vs APIs vs wrappers - when each approach actually makes sense

Trade-offs between different ways of accessing Gemini

Including lessons from Nix, Apple Containers, and lightweight setups without Docker

What works and what doesn’t running LLM tooling locally

Will agents replace search teams? A discussion

Daniel Tunkenlang

Principal, High Class Consultant

Agentic search consultant

What's the difference?
Where do we see agentic search fitting into user's journey? Does it replace the search bar?

Agentic search vs traditional search

What role do agents play on teams helping to solve relevance and query understanding problems?

Where agents fit into teams + planning

Where human taste, strategic thinking, intuition, and inspiration still fits on the search team

Where human judgment still matters

Cheat at Search Essentials: Evaluation, NDCG, and pals

Ex-Reddit, Ex-Shopify. Author: AI Powered Search+Relevant Search

How the best search teams evaluate and measure search relevance

Basics of search relevance evaluation

When to ignore traditional search metrics and just trust your A/B tests. How to interpret offline metrics.

Where practice diverges from theory

Do you use clicks? Human labels? LLM as a judge? What are the pros/cons of these approaches?

Pros / cons of different types of labeled relevance data

Cheat at Search Essentials: Vectors and Embeddings

Author. Builder. Search practitioner.

What is a 'vector embedding' and why do they capture meaning? And how can that be used to build a search system?

Learn how embeddings implement 'semantic search'

Why do we need a vector database? And why are they different from traditional search engines? Should they be different?

Tackle the practical realities vector retrieval

Offer a concrete and concise explanation of how you will
help students understand and apply this lesson.


How are vector + lexical techniques combined?

ReasoningLayer.ai - neurosymbolic reasoning for your data

David Loiret

Founder - ReasoningLayer.ai

Agentic Search Trainer + Consultant

How formal logic grounds AI systems to eliminate hallucinations and inconsistency.

Why LLMs Need Symbolic Reasoning to Be Reliable

How a shared ontology becomes a single source of truth that automatically drives agents, dashboards, and applications

How Executable Knowledge Replaces Manual AI Plumbing

Learn how logic and rules move off the LLM, cutting token costs while improving accuracy and auditability.

How Zero-Token Reasoning Cuts Cost and Improves Accuracy

Cheat at Search Essentials: BM25 + Lexical

Principal AI Engineer in Search

Learn the baseline that rules them all (BM25) and how it works

Basics of Information Retrieval

How the simple decision of how to break up strings can make or break search

The importance of tokenization

How to achieve semantic search without vector search - and why you might prefer a lexical semantic approach

Cheat at semantic search without the vectors

RAG isn't a vector search problem

John Berryman

Ex-Github Copilot; AI Consultant

Agentic Search Consultant

The marketing hype says you should deploy billion scale vector search before RAG. Is this true?

Rethink RAG

Bulid RAG with the retrieval systems you already have on hand

Solve RAG faster

Build a RAG with easier to manage lexical search. Using tried and true search methods.

Lexical RAG and more

Tuning the Tuners-Can agents automate OpenSearchs relevance?

Eric Pugh

Co-Founder, OpenSource Connections; OpenSearch Maintainer

Search Relevance Consultant

Improving search relevance algorithm requires careful coordination between agents and sub-agents

Bootstrapping a multi-agent optimization

Explore how deeply agents can grok search relevance. Do they get the task as well as the everyday human?

Can agents grok search relevance?

A review of the work needed to make OpenSearch's endpoints "agent-ready"

Making OpenSearch Agent Ready

SearchArray - let's rethink full-text search

Black belt search ninja space cowboy (and consultant)

Every search engine forces you to learn a DSL. Why we should be removing, not adding, to the cognitive load.

Why search engine DSLs continue to confuse users

How I built an in-memory full text search system good enough to recreate core search functionality around dataframe

How to build your own full-text search

Why representing search relevance in a dataframe creates maximum flexibility over a traditional API

A different perspective on search APIs

What is agentic search? (and why should I care!?)

A good agent can plan and reason, turning a user's query into a plan to use even basic search tools to explore a corpus

Let agents take control of your search

Traditional RAG gives you one chance to formulate a query and get the right information for the user.

How Agentic RAG wins over traditional RAG

An agent uses its own judgment on whether the returned content helps the user. Go beyond this with user input.

Cheat at Search with Agents

Cheat at Search with Agents

Courses

Lightning Lessons

Reviews