Advanced Topics in Practical AB Testing

5.0 (25)

·

4 Days

·

Cohort-based Course

Learn from a world-leading expert advanced topics in A/B testing. Ten topics in three days of 2 hours each. ** Warning: Advanced course *

This course is popular

6 people enrolled last week.

Previously at

Microsoft
Airbnb
Amazon

Course overview

Improve your A/B testing capabilities

We will cover advanced topics, not typically addressed in courses and books, and leave significant time to interact and answer questions.


Classes are small: usually about 15 people attending live.


The course assumes you already took https://bit.ly/ABCourseRonKohavi or have equivalent knowledge.

Who is this course for

01

Data science managers and scientists who understand the basics of A/B testing and want to better understand advanced topics

02

Program managers who want to advance their understanding of A/B testing

Prerequisites

  • Follow-on to flagship course

    This is designed as a follow-on to the flagship course: https://maven.com/kohavi/abtesting

  • Statistics of A/B Testing

    Good understanding of p-value misconceptions, statistical power, OEC.

What you’ll get out of this course

Review advanced topics

We will review and discuss practical advanced topics.

Click "Expand all modules" in the Course Syllabus below to see the details.

Discuss your own questions

We allocate time to discuss your own questions

** Warning: prior knowledge assumed

This course assumes you have taken https://bit.ly/ABCourseRonKohavi or have equivalent knowledge.

Course syllabus

4 live sessions • 4 lessons

Week 1

Jun 2—Jun 5

    Jun

    2

    Session 1

    Mon 6/23:00 PM—5:00 PM (UTC)

    Jun

    3

    Session 2

    Tue 6/33:00 PM—5:00 PM (UTC)

    Jun

    5

    Session 3

    Thu 6/53:00 PM—5:00 PM (UTC)

    Session 1

    1 item

    Session 2

    1 item

    Session 3

    1 item

Post-course

    Jun

    12

    Optional: Alumni event

    Thu 6/123:00 PM—4:00 PM (UTC)
    Optional

    Optional: Alumni event

    1 item

5.0 (25 ratings)

What students are saying

What people are said about the EXL 2024 workshop (precursor to this class)

        My personal highlight? Ron Kohavi's masterclass on experimentation and specially advanced statistical concepts...A key takeaway for those running experiments: Have you considered giving your results a "haircut"? If not, it's time to start. Doing so helps ensure you're not overstating your outcomes.
Hoang Nguyen vu

Hoang Nguyen vu

Director of Product, Sixt
        Ron Kohavi running an amazing advanced A/B testing masterclass
Annika Thompson

Annika Thompson

Managing director, Speero
        ...how amazing it is to listen to Ronny...it definitely allowed me to "walk the talk", and try something new, different, uncomfortable, which made me test my ability to be adaptable, curious, resilient! 
Anna Malka Campiani Gimenez

Anna Malka Campiani Gimenez

Founder & Experimentation Consultant, JOSI
        I attended the Austin workshop and thought it was great...I endorse and recommend.
A.J. Long

A.J. Long

Manager, product experimentation & Analytics, FanDuel

100% Customer Satisfaction Guarantee

Not satisfied after attending the first session and before session 2 starts? Get a full refund

Not satisfied after attending the first session and before session 2 starts? Get a full refund

A pattern of wavy dots

Join an upcoming cohort

Advanced Topics in Practical AB Testing

2 June 2025 - Advanced AB

$1,199

Dates

June 2—5, 2025

Payment Deadline

June 2, 2025

Don't miss out! Enrollment closes in 5 days

Get reimbursed

Meet your instructor

Dr. Ronny Kohavi

Dr. Ronny Kohavi

Ronny Kohavi was an executive at Amazon, Microsoft, and Airbnb and has over 20 years of experience running A/B tests and leading experimentation teams. He loves to teach, and his papers have over 55,000 citations. He co-authored the best-selling book: Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing (with Diane Tang and Ya Xu), which is a top-10 data mining book on Amazon. He is the most viewed writer on Quora's A/B testing and received the Individual Lifetime Achievement Award for Experimentation Culture in Sept 2020.


Ronny holds a PhD in Machine Learning from Stanford University.


See more at http://www.kohavi.com

Course schedule

Six hours: 3 days x 2-hour sessions

  • Monday, Tuesday, and Thursday

    8-10AM Pacific time


  • Optional Q&A

    15-30 minutes after each session

    Your opportunity to ask questions about the course material or anything else related to experimentation

Frequently Asked Questions

A pattern of wavy dots

Join an upcoming cohort

Advanced Topics in Practical AB Testing

2 June 2025 - Advanced AB

$1,199

Dates

June 2—5, 2025

Payment Deadline

June 2, 2025

Don't miss out! Enrollment closes in 5 days

Get reimbursed
Free resource

A/B Testing Myths Lightning Lesson

Many myths are based on some real limitation. The lesson reviews the claims and why they are usually myths in practice.


Recording is 35 minutes + 15 minutes of Q&A.


Example myths

Overstated effect, concurrent experiments are invalid, use pre-post analysis with few users, 95% confidence has 5% error.

Get this free resource

Free resource

The ultimate guide to A/B testing: Podcast with Lenny Rachitsky

The podcast has been viewed over 50,000 times.

Get it by entering your email

Free resource

Chapter 1 of Trustworthy Online Controlled Experiments: a Practical Guide to A/B Testing

Full text of Chapter 1 and table of contents for this best-selling book by the course instructor.

Get it by entering your email

100% Customer Satisfaction Guarantee

$1,199

5.0 (25)

·

5 days left to enroll