Free Lesson

Put Error Bars on Your LLM Metrics

Hosted by Bruno Gonçalves

Wed, Sep 23, 2026

6:00 PM UTC (30 minutes)

Virtual (Zoom)

Free to join

Invite your network

Go deeper with a course

Build a Production-Grade LLM Eval Harness
Bruno Gonçalves
View syllabus

What you'll learn

How to bootstrap a confidence interval on any metric

Resample your results a thousand times in twenty lines of Python. No distribution assumptions required.

How to put error bars on accuracy, cost, and latency

One method covers every number you report. Intervals turn single scores into ranges you can defend.

How to explain score swings between identical runs

Temperature, sampling, and judge variance move scores run to run. See the spread and stop chasing ghosts.

Why this topic matters

You ran the same eval twice and got 84.2, then 81.9. Which number goes in the report? Without error bars, every score is a coin flip dressed as a fact. Teams chase phantom regressions, celebrate phantom wins, and burn weeks on noise. The bootstrap fixes this with twenty lines of Python. Once your metrics carry intervals, your reports survive scrutiny and your decisions stop wobbling.

You'll learn from

Bruno Gonçalves

PhD physicist and corporate trainer.

I earned a PhD in the Physics of Complex Systems in 2008 and held a tenured faculty position at Aix-Marseille Université and served as a Data Science fellow at NYU’s Center for Data Science before moving to Industry.

Now I consult in Generative AI, Machine Learning, and Blockchain Analytics.

The teaching runs through it all. I run corporate trainings, and my published video courses cover NLP, data visualization, and time series analysis. My newsletter, Data For Science, reaches 4,000+ subscribers, and every post ships with a working notebook.

In my sessions, every claim gets a number. You leave with code that runs on Monday morning.

Previously at

JPMorgan Chase & Co.
TRM Labs
New York University
Data For Science
See all products from Bruno

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.