 Build a coding agent to optimize ML, performance, and more

http://softwaredoug.com

Reddit

Fiverr.com

Apple

AWS Experience

Shopify

Doug Turnbull

Search + ML Expert

Build your own Autoresearcher

Cheat at Search

August 2026

Using simple API calls to OpenAI, we'll walk through how simple it is to build a coding agent. That's the bedrock of our work. AutoResearch requires us to think about not just the code generation - but the constraints we put around it. You can do that with Claude Code, Pi, and other agents. But to learn, we'll do it ourselves.

Using simple API calls to OpenAI, we'll walk through how simple it is to build a coding agent. That's the bedrock of our work. 

AutoResearch requires us to think about not just the code generation - but the constraints we put around it. You can do that with Claude Code, Pi, and other agents. But to learn, we'll do it ourselves.

Build your own coding agent with simple API calls

Search ranking provides an accessible playground for building our autoresearcher. It's ML-ish, but not ML. We'll touch briefly on the open datasets we'll attempt to optimize with our autoresearcher.

Search ranking provides an accessible playground for building our autoresearcher. It's ML-ish, but not ML. We'll touch briefly on the open datasets we'll attempt to optimize with our autoresearcher.

Introduction to the problem: search ranking

Programatic guardrails help prevent the autoresearcher from overfitting and other suboptimal outcomes (ie too slow, etc). We'll add a framework for you to express your constraints - while also limiting obvious overfitting problems for our search problem.

Programatic guardrails help prevent the autoresearcher from overfitting and other suboptimal outcomes (ie too slow, etc). We'll add a framework for you to express your constraints - while also limiting obvious overfitting problems for our search problem.

Designing the agent to avoid overfitting

Take a break, come back, explore what our autoresearchers have done. We'll babysit our autoresearchers to ensure they're behaving. We'll check in on them and mutually share our findings at the end of the day.

Take a break, come back, explore what our autoresearchers have done. We'll babysit our autoresearchers to ensure they're behaving. We'll check in on them and mutually share our findings at the end of the day.

Let it rip!

We'll take turns sharing what we've built, and how far it's gone.

We'll take turns sharing what we've built, and how far it's gone.

Show and Tell

We'll see how insanely simple it is to ask OpenAI, Anthropic, etc to produce code

Build your own coding agent

We'll encourage the process to train towards an objective

Train to an objective

Ensure our objectives don't overfit and harm other crucial aspects of our code

Enforce guardrails

Optimizing models + code - long the realm of ML engineers and system engineers. We'll take an example problem (search relevance) and build a coding agent to optimize outcomes. Armed with these techniques, you'll be able to take on any range of problems from ML to performance.

Projects

Async content

Developers eager to learn how coding agents work from scratch

ML Practitioners who want to explore a new method of optimization

Data scientists excited to support autoresearch experts

Go back to course content and recordings whenever you need to.

Lifetime access

Stay accountable and share insights with like-minded professionals.

Community of peers

Share your new skills with your employer or on LinkedIn.

Certificate of completion

Build your own Autoresearcher by Doug Turnbull on Maven

Build a vector database from scratch

Shopify.com

Wikipedia

O'Reilly Media

Search Consultant and Coach

John Berryman

Founding member Github Copilot

Learn the most common graph-based vector search algorithm used in Elasticsearch, Weaviate, QDrant, Pinecone, etc

Learn how embedding similarity search works

As you watch an expert live-code, see where the algorithm falls apart as a real-life expert makes a mistake

See where vector search goes wrong

Dense vectors provide specific constraints on retrieval solutions - learn what these are, how they can go wrong

Deepen your vector search knowledge

Could your search be better without vectors? BM25 + friends

OpenSource Connections

Learn the baseline that rules them all (BM25) and how it works

Basics of Information Retrieval

How the simple decision of how to break up strings can make or break search

The importance of tokenization

How to achieve semantic search without vector search - and why you might prefer a lexical semantic approach

Semantic search without the vectors

Cheat at Search Essentials: Vectors and Embeddings

Ex-Reddit, Ex-Shopify Search

What is a 'vector embedding' and why do they capture meaning? And how can that be used to build a search system?

Learn how embeddings implement 'semantic search'

Why do we need a vector database? And why are they different from traditional search engines? Should they be different?

Tackle the practical realities vector retrieval

Offer a concrete and concise explanation of how you will
help students understand and apply this lesson.


How are vector + lexical techniques combined?

Cheat at Search Essentials: Evaluation, NDCG, and pals

Ex-Reddit, Ex-Shopify. Author: AI Powered Search+Relevant Search

How the best search teams evaluate and measure search relevance

Basics of search relevance evaluation

When to ignore traditional search metrics and just trust your A/B tests. How to interpret offline metrics.

Where practice diverges from theory

Do you use clicks? Human labels? LLM as a judge? What are the pros/cons of these approaches?

Pros / cons of different types of labeled relevance data

CTO's Playbook for Faster Agentic RAG

Qdrant

NVIDIA

Nirant Kasiwal

Creator FastEmbed

Led Search at Reddit, Shopify, Wikipedia

know exactly when to swap OpenAI for FastEmbed and improve both recall and latency.

Latency vs Recall Cheat-Sheet

live-coded: BM25 + vector ➜ tool-calling agent

Hybrid-to-Agentic Upgrade Path

7-point checklist that spots 90 % of real-world retrieval failures before users do.

Stealable RAG Audit Framework

Cheat at Search Essentials: BM25 + Lexical

LexisNexis

Principal AI Engineer in Search

Cheat at semantic search without the vectors

RAG is the what. Agentic Search is the how.

Amazon Web Services

Retrieval Strategist - Ex-Reddit, Ex-Shopify

RAG fails because we've been taught a myopic view that misses out on the lessons of Information Retrieval

Good RAG isn't chunking + vector databases

We think RAG means "LLMs that search". That's a half step. Real progress comes from search agents that self correct

Agentic Search gets RAG unstuck

From reasoning to building a good harness - the craft of building a good RAG runs through agentic search - not vector DB

How RAG should actually be built - from first principles

Building Deep Research Systems

Zillow

Google

eBay Inc.

Andrei Lopatenko

Search at Zillow, Google, eBay, Apple

Agentic Search Consultant

How retrieval augmentation and semantic grounding expand LLM capabilities for knowledge discovery.

Pushing beyond the LLMs knowledge

The design principles behind multi-hop reasoning and deep exploration architectures.

Deep Research Design Principles

Evaluation methods and frameworks shaping next-generation research and information systems.

The next generation of search eval

Navigating your tech career in the age of layoffs and AI

Daydream

Muck Rack

Team Lead at Shopify, Reddit, and AI Startups

Brian Pedersen

Principal, "The Search Bar"

What tools and skills should you acquire to thrive in with ubiquitious AI and looming layoffs?

How can I future-proof my career?

How to match up what makes you happy with the needs of the job market

What does fulfillment look like?

What skills do hiring managers focus on increasingly? What can help you standout on your team, or in the next interview?

What do hiring managers want?

What is agentic search? (and why should I care!?)

Search Consultant

A good agent can plan and reason, turning a user's query into a plan to use even basic search tools to explore a corpus

Let agents take control of your search

Traditional RAG gives you one chance to formulate a query and get the right information for the user.

How Agentic RAG wins over traditional RAG

An agent uses its own judgment on whether the returned content helps the user. Go beyond this with user input.

How to inject the user's perspective

Hybrid search live coded from scratch

Elastic

Led Search at Shopify, Reddit

Ben Trent

Sr. Principal Engineer Elastic

How a vector search index, like Elasticsearch's, implements filtering using algorithms to combine with lexical retrieval

Internals of hybrid search

See where hybrid search goes wrong

Agentic Search for Context Engineering

Leonie Monigatti

Sr. Developer Advocate, Elastic

Search + Snuggie Enthusiast

Understand how retrieval evolved from passive RAG pipelines to agents that build their own context with search tools

Why context engineering is 80% agentic search

Understand which search tools exist, how they work, and their trade-offs

The search tool landscape

Practical considerations for adding more search tools

How to curate your agent's search tools

2026 will be the year of agentic search

Vespa.Ai

Yahoo

Jo Kristian Bergum

Search + Retrieval Expert. CEO, Hornet.dev

Harnesses, smarter models, and reasoning all upgraded agentic capabilities. 2026 that upgrade will be search.

Retrieval will be the major upgrade to agents in 2026

Is 'vector search' done? Why models like late interaction / ColBERT seem poised to reinvent how agents think + search

The power of late interaction, not just 'vector search'

How retrieval - not agentic reasoning - becomes the primary factor in deep research

What is "BrowseComp Plus" benchmark?

Beyond chunks - how to actually data model for RAG

Radu Gheorghe

Software Engineer, Vespa.ai

Trade-offs between large documents with chunk arrays and chunk documents with denormalized metadata

Why preferring large documents works best for RAG

Combining chunk and document-level features into a single searchable entity

The ideal data model for RAG retrieval

Using rank profiles to compute chunk relevance within each document and return top N chunks for each

How to surface the most relevant chunks

Underrated: Elasticsearch for vector search + AI

Principal Consultant, SoftwareDoug LLC

Philipp Krenn

Head of DevRel at Elastic

How a search engine like Elasticsearch combines decades of Information Retrieval with vector retrieval innovations

Why you want a search engine, not just a vector DB

Learn about BBQ and other optimizations Elasticsearch has developed in vector retrieval

Elasticsearch's vector search innovations

How Elasticsearch's new features help target emerging use-cases like RAG

Elasticsearch for AI and RAG

Wikimedia Foundation

Wayfair

Author. Builder. Search practitioner.

RAG isn't a vector search problem

GitHub

Eventbrite

Ex-Github Copilot; AI Consultant

The marketing hype says you should deploy billion scale vector search before RAG. Is this true?

Rethink RAG

Bulid RAG with the retrieval systems you already have on hand

Solve RAG faster

Build a RAG with easier to manage lexical search. Using tried and true search methods.

Build your own Autoresearcher

Doug Turnbull

Build a coding agent to optimize ML, performance, and more

Workshop agenda

Build your own coding agent

Train to an objective

Enforce guardrails

Learn directly from Doug

Doug Turnbull

Who this workshop is for

What's included

Frequently asked questions