Dani

Customer Support

Series 4

Instructor 4

test

Learn from a world-leading expert how to design and analyze trustworthy A/B tests to evaluate ideas, integrate AI/ML, and grow your business

Author of best-selling A/B testing book

Microsoft

Airbnb

Amazon

Dr. Ronny Kohavi

Technical fellow and VP at Microsoft & Airbnb, co-author of A/B testing book

14 Sept 2026

9 Nov 2026

Through multiple real examples of well-run experiments and stories from Microsoft, Amazon, and Airbnb, you will see the humbling reality that we are terrible at assessing the value of ideas.Trivial changes can be surprisingly useful, whereas large efforts often fail. Accelerate innovation by building Minimum Viable Products (MVPs) and features, and make the organization more evidence-based and humble as it adopts and learns to use evidence from the gold standard in science—the controlled experiment.You will understand the challenges in designing and running trustworthy randomized controlled experiments (A/B tests), including the importance of the Overall Evaluation Criterion (OEC), scaling, common pitfalls, and Twyman's Law. We cover cultural challenges, institutional memory, maturity model, observational causal studies, offline evaluation, AI/Machine learning and triggering, build-vs-buy trade-offs, challenges, and requested topics. The Advanced Topics course is designed as a technical follow-on: <a target="_blank" rel="noopener noreferrer nofollow" href="https://bit.ly/AdvancedABFS">https://bit.ly/AdvancedABFS</a>

Empower your organization to be data-driven and innovative

Internalize the humbling reality: we are poor at assessing the value of ideas

Understand the key advantages and limitations of A/B testing

Learn how to design metrics and the Overall Evaluation Criterion

Learn how to design trustworthy A/B tests

Learn about the cultural challenges

Learn about the relationship to AI and Machine Learning Modeling

Data science managers and scientists will be able to design and interpret the experimental results in a trustworthy manner

Program managers focused on growth, revenue, conversions, and prioritization will understand how to provide the org with robust clear metric

Engineering leaders will be able to make organizations more data-driven and efficient with fewer severe incidents through A/B tests

Learn advanced A/B-testing topics from a world-leading expert. Ten topics in three days of 2 hours each.

⚠️ Warning: Advanced course 

5 Oct 2026

30 Nov 2026

We cover advanced topics not typically addressed in courses and books and leave significant time for interactions and Q&amp;A.Classes are small: typically ~15 live attendees. The course assumes you already took <a target="_blank" rel="noopener noreferrer nofollow" href="https://bit.ly/ABCourseRonKohavi">https://bit.ly/ABCourseRonKohavi</a> or have equivalent knowledge.

Improve your A/B testing capabilities and build trust

Improve your A/B Testing capabilities and trust

Discuss your own questions

Designed as a follow-on to the flagship course: https://maven.com/kohavi/abtesting. 

Take that course first, or have equivalent knowledge

Follow-on to flagship course

Good understanding of p-value misconceptions, statistical power, OEC.

Statistics of A/B Testing

Data science managers and scientists who understand the basics of A/B testing and want to better understand advanced topics

Product and program managers who want to advance their understanding of A/B testing

Students will gain awareness to recognize an emotional reaction and pause to improve decision, relationships, and outcomes.  

 Rewire your brain. Reimagine your life.

Living Well USA LLC | Los Gatos CA

Patty Purpur de Vries

Offering practical tools for more calm, clarity, meaning & joy.

Join the Founding Circle

As a member of the Founding Circle you will not simply complete a course. You will become a Founding Practitioner in a growing community committed to leading with awareness, emotional steadiness, discernment, connection, and humanity.

Your insights and experiences will help establish the foundation for future cohorts and for an ongoing community of those who wish to continue living and leading through in the practice to:

Observe. Honor. Discern. Choose. Connect. Create.

Founding Circle

This is the second cohort in the Self-Leadership Series.  I hope you'll join us.

Trail Blazers

If you are seeking self-regulation under pressure, this course is for you.

From conditioned reaction to conscious creation.

Stay grounded and think more clearly under pressure

Build trust and psychological safety within teams

You don't build the AI. You decide what happens next.

Research · Data · Program Leadership

PayPal

Intuit

Sephora

S C Johnson

Dave Rogers

Research & analytics leader helping teams make better AI decisions

2026 Fall

A practical, live workshop for AI program managers, technical project managers, product leaders, and Chiefs of Staff who need to assess AI performance claims, challenge the evidence, and recommend what to do next — without needing to build or train a model.Learn to distinguish convincing numbers from trustworthy evidence and turn evaluation results into decisions about what to ship, change, investigate, or stop.

Learn to ask better questions, recognize trustworthy AI evidence, and make confident, evidence-based decisions about what happens next.

Understand What's Being Measured

Assess Whether Evidence Is Trustworthy

Challenge AI Performance Claims

Turn Evidence Into Decisions

Speak the Language of AI Teams

Walk Into Your Next AI Review Prepared

Product, program, and technical program managers who review AI results they didn't produce and need to decide what happens next.

Chiefs of Staff, operations, and business leaders asked to approve, fund, or expand an AI pilot.

Data, analytics, research and insights professionals who assess AI performance and turn findings into recommendations for teams and leaders.

Series 4

What you'll learn

Why this topic matters

You'll learn from

Instructor 4

Dani