Lightning Lessons

De-Risking LLM Model Switches w Evals & Prompt Optimization

Hosted by Amir Feizpour and Hugo Mailhot

Fri, Oct 3, 2025

4:00 PM UTC (45 minutes)

Virtual (Zoom)

Free to join

Invite your network

Go deeper with a course

Build Multi-Agent Applications - A Bootcamp - LangGraph, Cursor, n8n
Amir Feizpour, PhD
View syllabus

What you'll learn

Establishing Reliable Evals

How consistent evaluation frameworks reduce risk in high-stakes LLM migrations.

Automated Prompt Optimization

Using evals as a foundation to systematically improve prompts and model behavior.

Scaling RAG Across Models

Lessons from migrating Coveo’s flagship RAG product to new LLMs multiple times per year.

Why this topic matters

I'll be talking about LLM evals best practices in use at Coveo, and how we use them as the foundation for automated prompt optimization. We used the above many times to migrate Coveo's flagship RAG product (Coveo Relevance Generative Answering) to new LLM models, an unavoidable and high-risk project that we have to go through a few times a year.

You'll learn from

Amir Feizpour

Founder @ Aggregate Intellect

Amir Feizpour is the founder, CEO, and Chief Scientist at Aggregate Intellect building a generative business brain for service and science based companies. Amir has built and grown a global community of 5000+ AI practitioners and researchers gathered around topics in AI research, engineering, product development, and responsibility. Prior to this, Amir was an NLP Product Lead at Royal Bank of Canada. Amir held a research position at University of Oxford conducting experiments on quantum computing resulting in high profile publications and patents. Amir holds a PhD in Physics from University of Toronto. Amir also serves the AI ecosystem as an advisor at MaRS Discovery District, works with several startups as fractional chief AI officer, and engages with a wide range of community audiences (business executives to hands-on developers) through training and educational programs. Amir leads Aggregate Intellect’s R&D via several academic collaborations.

Hugo Mailhot

CoreNLP Lead @ Coveo

Hugo Mailhot is a researcher and team lead specializing in large language models and natural language processing. He has worked across academia, finance, and industry, with experience in quantitative trading, data science leadership, and AI product development. Currently at Coveo, Hugo leads the CoreNLP team, helping evolve and strengthen the company’s LLM-powered offerings. He is passionate about bridging research and practice, and brings a background that spans linguistics, machine learning, and applied AI.

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.