The advantage of a live course is the benefit of hearing the real memorable stories, many that don't make it to books or articles, chosen from over 20 years of experimentation: motivation, pushback, success/failure rates

Understand the key benefits of controlled experiments, or A/B tests, including causality, surprising examples, metrics, interpreting results, trust and pitfalls, Twyman's law, and A/A tests

Designing metrics is hard. There is a hierarchy of metrics and perverse incentives. The most important metrics comprise of the OEC - The Overall Evaluation Criterion. We will look at good and bad examples.

Getting numbers is easy; getting numbers you can trust is hard. We'll discuss pitfalls and how to build a reliable and trustworthy experimentation system.

Learn about the cultural challenges, the humbling results (most ideas fail, pivoting, iterating, learning), institutional memory, ideation, prioritization, experimentation platforms

When building AI or machine learning models, using A/B testing and triggering to evaluate the models that were built offline based on historical data

The course focuses on developing the intuition and common misunderstandings, without the details of the statistics, which you can find in many books. We cover p-values, statistical power, and triggering.

You can go as technical as you want in the Q&A and community discussions

When you can't run an A/B test, quasi-experimentation methods and the risks of observational causal studies

What are key challenges in the field

If there is something specific you want to cover, there is time allocated for that