A/B Testing Myths

Hosted by Ronny Kohavi

Thu, Feb 20, 2025

4:00 PM UTC (45 minutes)

Virtual (Zoom)

Free to join

229 students

Invite your network

Go deeper with a course

Accelerating Innovation with AB Testing
Dr. Ronny Kohavi

What you'll learn

A/B testing myths

Many myths are based on some real limitation. We will review the claim and why it is usually a myth in practice

Example myths

Overstated effect, concurrent experiments are invalid, use pre-post analysis with few users, 95% confidence has 5% error

Suggest and vote on myths

https://bit.ly/ABTestingMythsSuggestions

Why this topic matters

A/B testing has transformed many organizations and helped accelerate innovation, but there are calls to limit or stop A/B testing based on myths

You'll learn from

Ronny Kohavi

Technical fellow and VP at Microsoft & Airbnb, co-author of A/B testing book

Ronny Kohavi was an executive at Microsoft, Airbnb, and Amazon, and has over 20 years of experience running A/B tests and leading experimentation teams. He loves to teach, and his papers have over 60,000 citations. He co-authored the best-selling book: Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing (with Diane Tang and Ya Xu), which is a top-10 data mining book on Amazon. He is a most viewed writer on Quora's A/B testing and received the Individual Lifetime Achievement Award for Experimentation Culture in Sept 2020.


Ronny holds a PhD in Machine Learning from Stanford University.

See more at http://www.kohavi.com


Suggestions? Add and vote at https://bit.ly/ABTestingMythsSuggestions

Worked at

Microsoft
Amazon
Airbnb

Learn directly from Ronny Kohavi

By continuing, you agree to Maven's Terms and Privacy Policy.

© 2025 Maven Learning, Inc.