AI Product Engineering

Wed Jun 24·7:00 PM UTC

Your AI product does something dumb. How do you fix it? Most teams guess, or dump everything into an AI and trust whatever comes back, then ship fixes that don't move anything. This session is about the alternative: a hands-on way to find what's actually broken in your product, and to put AI to work on the problem without letting it fool you. You'll leave knowing how to debug your AI for real.

You'll learn from

Shreya Shankar

AI researcher; incoming CS Professor at Carnegie Mellon

Hamel Husain

ML Engineer with 20+ years of experience

Thu Jul 2·6:00 PM UTC

A capable model can still feel sluggish in production. The lag usually lives in the inference path: how requests queue and batch, prefill versus decode, the KV cache, and whether you stream tokens. Abi does this for a living. She'll show you where the latency comes from, and patterns that make agentic systems feel faster while costing less. There's a hands-on exercise so you can see it yourself.

You'll learn from

Abi Aryan

Founder of Abide AI; author of O'Reilly's LLMOps

Hamel Husain

ML Engineer with 20+ years of experience

Thu Jul 9·6:00 PM UTC

Single-vector retrieval is cheap, but it throws away detail that matters for hard queries. Late interaction keeps more of that detail, but the production cost is large. This lesson shows how we scaled it to billions of documents while keeping latency, filtering, and index updates practical.

You'll learn from

Marek Galovic

CEO, Co-Founder @TopK. ex-Pinecone, ex-Shopify

Hamel Husain

ML Engineer with 20+ years of experience

Thu Jul 9·7:00 PM UTC

Most teams have one person who really gets evals, and everything bottlenecks on them. Lucas took Hamel's open-source eval skills and turned them into something his whole team could run. His team uses evals to optimize the WhatsApp assistant Nova Escola ships to teachers across Brazil. We'll show you exactly how, so you can do it on your own product.

You'll learn from

Hamel Husain

ML Engineer with 20+ years of experience

Lucas Machado Rocha

AI Product Manager at Nova Escola

Fri Jul 10·5:00 PM UTC

How to choose an OCR Model

OCR can start as a simple API call, and that is often the right choice. But teams hit limits when they need better structure, custom outputs, lower cost, or more control. Joe Barrow will show when to use a managed OCR API, what a manageable OCR stack looks like, and what open models make possible.

You'll learn from

Joe Barrow

Senior Research Scientist at Adobe Research

Hamel Husain

ML Engineer with 20+ years of experience

Mon Jul 13·6:00 PM UTC

What Makes a Good Search Agent?

Search agents query, retrieve, and reason over external sources to answer knowledge-heavy questions. The hard part is knowing whether one is any good. Nandan will show you how to build benchmarks that measure search-agent quality, how to synthesize training data on a budget, and how to use trajectory analysis to debug these agents.

You'll learn from

Nandan Thakur

Creator of the BEIR and MIRACL benchmarks; PhD, University of Waterloo

Hamel Husain

ML Engineer with 20+ years of experience

Wed Jul 15·7:00 PM UTC

Can AI Actually Answer Your Data Questions?

Asking business questions of your data is the hottest enterprise AI use case, and one of the hardest to evaluate. Shreya Shankar & her colleagues built a benchmark that emulates real business scenarios, replicating what trips agents up: scattered systems, messy join keys, answers buried in text. She will share strategies you can use immediately in your workflows to improve performance.

You'll learn from

Shreya Shankar

AI researcher; incoming CS Professor at Carnegie Mellon

Hamel Husain

ML Engineer with 20+ years of experience

Fri Jul 24·7:00 PM UTC

How To Use Open Models Effectively

Frontier model prices keep climbing while open models keep getting better. For many types of production work an open model is good enough, and switching can cut your bill without hurting quality. Zach has taught thousands of engineers to run these models. He will show you when open models are the right call, and how to put them to work.

You'll learn from

Zach Mueller

Head of Developer Relations at Lambda (ex-Hugging Face)

Hamel Husain

ML Engineer with 20+ years of experience

Tue Jul 28·12:00 AM UTC

Why We Embed Documents, and Why You Should Go Multi-Vector

Almost everyone reaches for single-vector embeddings, then hits a wall: they dilute long documents and fall apart out-of-domain. Retrieval is about picking the right tool for the moment; single-vector methods no longer fit. Ben Clavié built some of the tools that made multi-vector retrieval usable. He'll zoom out on why he reaches for these models, and the measurable impact they bring to agents.

You'll learn from

Ben Clavié

Member of Technical Staff at Mixedbread

Hamel Husain

ML Engineer with 20+ years of experience

Wed Jul 29·4:00 PM UTC

Stop picking embedding models off the MTEB leaderboard

It's tempting to pick an embedding model according to which one tops the MTEB leaderboard. But MTEB doesn't tell you how the model will do on your data, where you'll often need to fine-tune it, or how it performs on speed and quality once you change quantization and dimensions. Radu shows how to juggle these variables so you get the most for your money.

You'll learn from

Radu Gheorghe

Software Engineer, Vespa.ai

Hamel Husain

ML Engineer with 20+ years of experience

Wed Jul 29·7:00 PM UTC

The cold-start eval problem: evals for new AI products

If you have no users and no data yet it can feel like "looks fine to me" is all you've got. Hamel and Shreya show you how to build evals from day zero, so every change you ship is one you can measure.

You'll learn from

Shreya Shankar

AI researcher; incoming CS Professor at Carnegie Mellon

Hamel Husain

ML Engineer with 20+ years of experience

Thu Jul 30·5:00 PM UTC

Stop Paying Full Price for LLM Classification

A lot of LLM products spend a fortune sending every request to an expensive model. Often the model is making a simple decision, like which model should handle a request or whether a document is relevant. These are classification problems, and because you can check the answer, you can make them cheaper and faster without losing quality. We'll show you how, on real data.

You'll learn from

Shreya Shankar

AI researcher; incoming CS Professor at Carnegie Mellon

Hamel Husain

ML Engineer with 20+ years of experience

Thu Jul 30·9:30 PM UTC

Don't build agents, build environments instead

Coding agents run on a machine you have to set up with the right code and tools. This session shows you how to make that machine start in seconds and stay reliable while the agent works. Adam builds this at Modal and will walk you through what works in production.

You'll learn from

Adam Azzam

Member of Product Staff at Modal

Hamel Husain

ML Engineer with 20+ years of experience

Fri Jul 31·5:00 PM UTC

Turn Eval Results Into a Better Model

Environments are the foundation for evaluating and training models and agentic systems. They define the tasks, provide the context and tools, and determine how success is measured. Florian and Will from Prime Intellect will show you how to build effective environments, avoid common pitfalls, and use them to evaluate and train stronger models.

You'll learn from

Will Brown

Research Lead at Prime Intellect

Florian Brand

Research Engineer at Prime Intellect

Hamel Husain

ML Engineer with 20+ years of experience