Could your search be better without vectors? BM25 + friends

Lightning Lessons

Practical skills & tools to accelerate your career

Could your search be better without vectors? BM25 + friends

Hosted by Doug Turnbull

849 students

What you'll learn

Basics of Information Retrieval

Learn the baseline that rules them all (BM25) and how it works

The importance of tokenization

How the simple decision of how to break up strings can make or break search

Semantic search without the vectors

How to achieve semantic search without vector search - and why you might prefer a lexical semantic approach

Why this topic matters

It's often said with chat interfaces and RAG, search has become the hard problem. Search has a long history and means more than vector databases. Let's learn how BM25 and similar techniques compliment your vector database. And why you might not want to always reach for embeddings.

You'll learn from

Doug Turnbull

Doug Turnbull is an expert in search technology and relevance engineering, currently serving as Principal Engineer at Daydream, where he builds hybrid search systems combining lexical and vector retrieval, and develops LLM-driven quality programs for e-commerce search. Previously, he led machine-learning-driven search initiatives at Reddit, significantly improving search relevance through Learning to Rank methods. Doug also advanced e-commerce search at Shopify and served as CTO at OpenSource Connections. He co-authored the influential book Relevant Search (Manning, 2016) and created popular open-source tools, including Quepid and the Elasticsearch Learning to Rank plugin. He regularly speaks at industry conferences, making search relevance accessible to engineers.