Cheat at Search with Agents

Nov 2025

Cohort 1

October 2025

Feb 2026

July 2025

June 2026

Rethink search to put tireless agents at work for human users

http://softwaredoug.com

Doug Turnbull

Snuggie and Search Enthusiast

Product Manager

Principal Engineer

Indie Consultant

Software Engineer

Principal recommender engineer 

Search Consultant

Generative AI Engineer

Software Engineer Search

Engineering Manager

Humans increasingly search via chat, not search bars. Agentic search goes beyond RAG to incorporate agentic reasoning into the search process. Agents iterate, refine, and learn to use search tools in away monolithic RAG fails to.Come to this class to:<ul><li>Leverage agentic reasoning to go beyond RAG - Traditional RAG can't reason. An agent learns about retrieval systems and tailors its searches for more accurate results. Go beyond naive RAG towards agentic search</li><li>Craft effective retrieval - Focus an agent's reasoning on the user's task, not "search", with task-targeted search tools. Tools built for agents; not for humans.</li><li>Go beyond LLM as a judge - Integrate human and clickstream feedback to guide the agent's judgment towards what users want</li><li>Improve keyword search - Take lessons learned through agentic search back to traditional "search bar" search, simplifying query and content understanding</li></ul>Come learn alongside search industry expert <a target="_blank" rel="noopener noreferrer nofollow" href="http://softwaredoug.com">Doug Turnbull</a> as we unpack how agents will change the search industry.This class is a fun and experimental course, focused on play and experimentation. It's part of my quest to find new, practical frontiers in Information Retrieval. I hope you can join me!

Agents are the new search engine. Go beyond RAG and keywords.

What makes a great search tool to an agent?

Use LLMs to help agents structure, organize, and find content

Agentic results without the cost

Orienting agentic search towards the human, not the machi

We will build on top of existing, core search knowledge

Lexical Search basics (do you know what BM25 is)

We will use common search datasets that use metrics like NDCG, etc to measure search

Basic search evaluation (judgments + metrics)

Vector search will be one tool in our toolkit we'll apply to the agentic tools that we build

Vector search basics (do you know what an embedding is?)

Search technologists that want to learn how to ensure RAG and agentic search win

Startups eager to get a fast start with a modern approach to search, sidestepping old, outdated practices

Data scientists that want to consider how agentic / LLM / RAG based search can be oriented towards user feedback and business outcomes

F*CK AGENTS! Let's build a workflow instead

Hugo Bowne-Anderson

AI & data engineer, consultant, educator of 6+ million students (ex-Yale)

Retrieval Consultant + Snuggie Connoisseur

Learn the common failure modes of autonomous agents, including drift, hidden errors, and evaluation blind spots

Why fully autonomous agents often fail in production

Use a practical decision framework to determine the right level of autonomy, supervision, and control for your AI system

When to use agents vs structured workflows

Discover a simple evaluation and observability pattern you can add immediately to measure performance and catch failures

A lightweight evaluation harness you can implement now

Build a vector database from scratch

Search Consultant and Coach

John Berryman

Founding member Github Copilot

Learn the most common graph-based vector search algorithm used in Elasticsearch, Weaviate, QDrant, Pinecone, etc

Learn how embedding similarity search works

As you watch an expert live-code, see where the algorithm falls apart as a real-life expert makes a mistake

See where vector search goes wrong

Dense vectors provide specific constraints on retrieval solutions - learn what these are, how they can go wrong

Deepen your vector search knowledge

Could your search be better without vectors? BM25 + friends

Learn the baseline that rules them all (BM25) and how it works

Basics of Information Retrieval

How the simple decision of how to break up strings can make or break search

The importance of tokenization

How to achieve semantic search without vector search - and why you might prefer a lexical semantic approach

Semantic search without the vectors

Cheat at Search Essentials: Evaluation, NDCG, and pals

Ex-Reddit, Ex-Shopify. Author: AI Powered Search+Relevant Search

How the best search teams evaluate and measure search relevance

Basics of search relevance evaluation

When to ignore traditional search metrics and just trust your A/B tests. How to interpret offline metrics.

Where practice diverges from theory

Do you use clicks? Human labels? LLM as a judge? What are the pros/cons of these approaches?

Pros / cons of different types of labeled relevance data

Cheat at Search Essentials: Vectors and Embeddings

Ex-Reddit, Ex-Shopify Search

What is a 'vector embedding' and why do they capture meaning? And how can that be used to build a search system?

Learn how embeddings implement 'semantic search'

Why do we need a vector database? And why are they different from traditional search engines? Should they be different?

Tackle the practical realities vector retrieval

Offer a concrete and concise explanation of how you will
help students understand and apply this lesson.


How are vector + lexical techniques combined?

CTO's Playbook for Faster Agentic RAG

Nirant Kasiwal

Creator FastEmbed

Led Search at Reddit, Shopify, Wikipedia

know exactly when to swap OpenAI for FastEmbed and improve both recall and latency.

Latency vs Recall Cheat-Sheet

live-coded: BM25 + vector ➜ tool-calling agent

Hybrid-to-Agentic Upgrade Path

7-point checklist that spots 90 % of real-world retrieval failures before users do.

Stealable RAG Audit Framework

Cheat at Search Essentials: BM25 + Lexical

Principal AI Engineer in Search

Cheat at semantic search without the vectors

Building Deep Research Systems

Andrei Lopatenko

Search at Zillow, Google, eBay, Apple

Agentic Search Consultant

How retrieval augmentation and semantic grounding expand LLM capabilities for knowledge discovery.

Pushing beyond the LLMs knowledge

The design principles behind multi-hop reasoning and deep exploration architectures.

Deep Research Design Principles

Evaluation methods and frameworks shaping next-generation research and information systems.

The next generation of search eval

Navigating your tech career in the age of layoffs and AI

Team Lead at Shopify, Reddit, and AI Startups

Brian Pedersen

Principal, "The Search Bar"

What tools and skills should you acquire to thrive in with ubiquitious AI and looming layoffs?

How can I future-proof my career?

How to match up what makes you happy with the needs of the job market

What does fulfillment look like?

What skills do hiring managers focus on increasingly? What can help you standout on your team, or in the next interview?

Cheat at Search with Agents

Cheat at Search with Agents

Courses

Lightning Lessons

Reviews