Build Multi-Agent Systems You Can Audit

Part of The AI Evaluation Handbook

Hosted by Stefan Jansen

Wed, Jun 24, 2026

3:00 PM UTC (30 minutes)

Virtual (Zoom)

Free to join

Invite your network

Go deeper with a course

Building Multi-Agent Forecasting Systems
Stefan Jansen
View syllabus

What you'll learn

What makes a forecast scoreable, not just persuasive

Five stages — question, evidence, independent runs, aggregation, scoring — that decide whether your number holds up.

Audit while the agent works, not after

Replayable evidence, independent runs, explicit aggregation — captured as the trace forms, not bolted on at the end.

The result most agent demos hide

Brier, log score, calibration locked before resolution — including when agent + consensus beats either alone.

Why this topic matters

Most AI forecasts collapse under one question: "Why should I trust this number?" Auditability isn't bolted on at the end — it's decided five stages earlier: the question, the evidence, the independent runs, the aggregation rule, and the scoring plan locked before resolution. Build the process; the audit takes care of itself.

You'll learn from

Stefan Jansen

Author, ML for Trading · Founder, Applied AI · Investing since 2013

Stefan is the author of ML for Trading — the book and open-source companion code (19,000+ GitHub stars) that have become a practitioner reference for applying ML to financial markets. The 2026 third edition expands to nine cross-asset case studies, with a foreword by Antonio Gulli, Senior Director, Google. He maintains the Zipline fork the quant community relies on, and built the six-library stack — data to live — behind the third edition's case studies. Investment partner since 2013, he has built trading platforms and live strategies across asset classes. In 2016 he founded Applied AI, which brings production ML to investment teams and other data-rich verticals. He has taught ML to 110,000+ professionals through DataCamp and General Assembly, incl. at Bloomberg and BlackRock.
See all products from Stefan

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.