Mastering LLM Application Testing

Hosted by Hugo Bowne-Anderson and Stefan Krawczyk

Share this lesson

673 students

What you'll learn

Write effective pytest cases for testing LLM outputs.

Build a framework for iterative improvement of LLM apps

Evaluate LLM results systematically to improve stability

Why this topic matters

LLM-powered applications require constant iteration and evaluation to ensure robust and reliable outputs. In this Lightning Lesson, you’ll learn how to apply pytest to systematically test and refine your LLM apps. We’ll cover how to identify and address failure modes, evaluate outputs, and build confidence in your app’s performance.

You'll learn from

Hugo Bowne-Anderson

Podcaster, Educator, DS & ML expert

Hugo Bowne-Anderson is an independent data and AI consultant with extensive experience in the tech industry. He is the host of the industry Vanishing Gradients, where he explores cutting-edge developments in data science and artificial intelligence. As a data scientist, educator, evangelist, content marketer, and strategist, Hugo has worked with leading companies in the field. His past roles include Head of Developer Relations at Outerbounds, a company committed to building infrastructure for machine learning applications, and positions at Coiled and DataCamp, where he focused on scaling data science and online education respectively. Hugo's teaching experience spans from institutions like Yale University and Cold Spring Harbor Laboratory to conferences such as SciPy, PyCon, and ODSC. He has also worked with organizations like Data Carpentry to promote data literacy. His impact on data science education is significant, having developed over 30 courses on the DataCamp platform that have reached more than 3 million learners worldwide. Hugo also created and hosted the popular weekly data industry podcast DataFramed for two years. Committed to democratizing data skills and access to data science tools, Hugo advocates for open source software both for individuals and enterprises.

Stefan Krawczyk

13+years in MLOps: Ex-Stitch Fix, Ex-Nextdoor, Ex-LinkedIn

Stefan Krawczyk is the co-founder and CEO of DAGWorks, an open-source company driving two projects: Hamilton & Burr, whose mission to empower developers to build reliable AI agents & applications. He is a Y Combinator alum, StartX alum, and a Stanford graduate with a Master of Science in Computer Science with Distinction in Research. He has over thirteen years of experience in building and leading data & ML-related systems and teams, at companies like Stitch Fix, Idibon, Nextdoor, and Linkedin, his passion is to make others more successful with data by bridging the engineering gap between data science, machine learning, artificial intelligence, and the business.

Built, shipped, and lectured at:

LinkedIn
Stitch Fix
Stanford University
Outerbounds
Yale University
© 2025 Maven Learning, Inc.