A/B Testing Myths

Lightning Lessons

Practical skills & tools to accelerate your career

A/B Testing Myths

Hosted by Ronny Kohavi

739 students

What you'll learn

A/B testing myths

Many myths are based on some real limitation. We will review the claim and why it is usually a myth in practice

Example myths

Overstated effect, concurrent experiments are invalid, use pre-post analysis with few users, 95% confidence has 5% error

Suggest and vote on myths

https://bit.ly/ABTestingMythsSuggestions

Why this topic matters

A/B testing has transformed many organizations and helped accelerate innovation, but there are calls to limit or stop A/B testing based on myths

You'll learn from

Ronny Kohavi

Technical fellow and VP at Microsoft & Airbnb, co-author of A/B testing book

Ronny Kohavi was an executive at Microsoft, Airbnb, and Amazon, and has over 20 years of experience running A/B tests and leading experimentation teams. He loves to teach, and his papers have over 60,000 citations. He co-authored the best-selling book: Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing (with Diane Tang and Ya Xu), which is a top-10 data mining book on Amazon. He is a most viewed writer on Quora's A/B testing and received the Individual Lifetime Achievement Award for Experimentation Culture in Sept 2020.

Ronny holds a PhD in Machine Learning from Stanford University.

See more at http://www.kohavi.com

Suggestions? Add and vote at https://bit.ly/ABTestingMythsSuggestions

Worked at