Manisha Arora

Cracking A/B Testing Interviews

Siddarth R

Director of Data Science, Microsoft

Deep dive into the key technical concepts testing in A/B Testing Interviews

Key Focus Areas in A/B Testing Interviews

Learn the key concepts that interviewers are evaluating you on during AB Testing Interviews

How Interviewers Evaluate A/B Testing Answers

A framework to confidently design AB Tests and move from business objectives to metrics, design, and decisions.

A Clear Framework to Structure Any A/B Testing Question

Learn from a world-leading expert how to design and analyze trustworthy A/B tests to evaluate ideas, integrate AI/ML, and grow your business

Author of best-selling A/B testing book

Microsoft

Airbnb

Amazon

Dr. Ronny Kohavi

Technical fellow and VP at Microsoft & Airbnb, co-author of A/B testing book

9 March 2026

1 June 2026

Through multiple real examples of well-run experiments and stories from Microsoft, Amazon, and Airbnb, you will see the humbling reality that we are terrible at assessing the value of ideas.Trivial changes can be surprisingly useful, whereas large efforts often fail. Accelerate innovation by building Minimum Viable Products (MVPs) and features, and make the organization more evidence-based and humble as it adopts and learns to use evidence from the gold standard in science—the controlled experiment.You will understand the challenges in designing and running trustworthy randomized controlled experiments (A/B tests), including the importance of the Overall Evaluation Criterion (OEC), scaling, common pitfalls, and Twyman's Law. We cover cultural challenges, institutional memory, maturity model, observational causal studies, offline evaluation, AI/Machine learning and triggering, build-vs-buy trade-offs, challenges, and requested topics. The Advanced Topics course is designed as a technical follow-on: <a target="_blank" rel="noopener noreferrer nofollow" href="https://bit.ly/AdvancedABFS">https://bit.ly/AdvancedABFS</a>

Empower your organization to be data-driven and innovative

Internalize the humbling reality: we are poor at assessing the value of ideas

Understand the key advantages and limitations of A/B testing

Learn how to design metrics and the Overall Evaluation Criterion

Learn how to design trustworthy A/B tests

Learn about the cultural challenges

Learn about the relationship to AI and Machine Learning Modeling

Data science managers and scientists will be able to design and interpret the experimental results in a trustworthy manner

Program managers focused on growth, revenue, conversions, and prioritization will understand how to provide the org with robust clear metric

Engineering leaders will be able to make organizations more data-driven and efficient with fewer severe incidents through A/B tests

Learn advanced A/B-testing topics from a world-leading expert. Ten topics in three days of 2 hours each.

⚠️ Warning: Advanced course 

23 March 2026

15 June 2026 - Advanced

We cover advanced topics not typically addressed in courses and books and leave significant time for interactions and Q&amp;A.Classes are small: typically ~15 live attendees. The course assumes you already took <a target="_blank" rel="noopener noreferrer nofollow" href="https://bit.ly/ABCourseRonKohavi">https://bit.ly/ABCourseRonKohavi</a> or have equivalent knowledge.

Improve your A/B testing capabilities and build trust

Improve your A/B Testing capabilities and trust

Discuss your own questions

Designed as a follow-on to the flagship course: https://maven.com/kohavi/abtesting. 

Take that course first, or have equivalent knowledge

Follow-on to flagship course

Good understanding of p-value misconceptions, statistical power, OEC.

Statistics of A/B Testing

Data science managers and scientists who understand the basics of A/B testing and want to better understand advanced topics

Product and program managers who want to advance their understanding of A/B testing

Take the guesswork out. Select and deliver stories that convey Senior impact.

Ex-Staff Engineer turned Career Coach

Apple

NVIDIA

Thumbtack

Shine Garg

Ex-Staff Engineer turned career coach for Senior through Principal Engineers

Cohort Q1 2026

Cohort Q2 2026

You walk out of a behavioral interview knowing you have strong experience, but unsure how it landed. Maybe you rambled, froze, or gave a surface-level answer. The ambiguity and unclear expectations leave you anxious about rejection or being down-leveled.This course removes that ambiguity.Gain the confidence and peace of mind to ace behavioral interviews without memorizing answers or making them up on the fly.You’ll learn how to:✅ Decode what behavioral questions really assess✅ Select the right story in &lt; 30 sec✅ Use the STAR++ framework to convey senior-level thinking✅ Deliver answers confidently, like a peer✅ Ask strategic questions that reinforce your seniority and reveal real work cultureThis course distills the interview judgment I developed over 15 years, including:<ul><li>Upleveling from Senior to Staff</li><li>Securing an additional $500k in equity after a single behavioral round with a VP</li><li>Advancing directly to a Meta E6 onsite after an initial behavioral interview</li></ul>Join live instruction on my frameworks, guided exercises, peer practice with feedback, and a practical handout mapping common behavioral questions to hiring signals and anchor stories from my career to model your own answers.

Decode what hiring managers really evaluate. Consistently select, structure, and deliver stories that articulate Senior-level impact.

How behavioral interviews are used to *level* Senior engineers

The Signals that make Senior engineers hireable

How to structure compelling stories using STAR++

How to deliver your stories with the confidence of a peer, not a candidate

The strategic questions you can ask to uncover the 'real' work culture

A repeatable method for future interview preparation

You're a Senior Software Engineer or a mid-level engineer looking to up-level to Senior through interviews.Other job titles: See FAQs

You're a Senior IC in adjacent functional roles like AI/ML, Frontend, Infrastructure, Mobile, SRE, etc.

You struggle with selecting the right stories, structuring them to convey impact, answering follow-up questions, or managing nerves.

Learn Advanced JTBD quantitative methods that prioritize opportunity, guide product roadmaps, and shape exceptional customer experiences.

Managing Partner - Customer Centric LLC

Ecobee

lululemon athletica

G Adventures

Citi

Humana

JOHN GUSIFF

Insights Architect, Consultant CX/UX/MR

Intuit

ServiceNow

YouTube

DOMINIC RICCHETTI

Insights Architect, Consultant CX/UX/MR Strategy and Methods

Cohort 1

Cohort 2

🔍 Uncover. 📊 Quantify. 🚀 Innovate.Transform Your Product Experience and Strategy with Jobs-to-Be-Done ResearchMost teams stop at the customer interview. But understanding customer progress isn’t just about rich stories — it’s about translating that insight into an actionable, quantified strategy. Gain the tools, methods, and confidence to drive smarter product decisions leveraging proven jobs-to-be-done quantitative research methods.It will be case study-based course you'll get hands-on experience in the following:🎯 Define your market (job performer + job=to-be-done) using the Market Definition Canvas🔍 Identify the core job, related jobs, and emotional context🛠️ Create a robust process map to reveal friction and opportunity across the customer journey🎯 Pinpoint and validate the right target audience📋 Design outcome-based surveys that quantify product and CX opportunities📈 Run opportunity scoring to prioritize what matters most🧠 Make strategic decisions based on actual customer progress — not opinionsWe’ll leverage powerful frameworks and methods from Tony Ulwick and Jim Kalbach.

Uncover. Quantify. Innovate.
Transform Your Product and CX Strategy 

Frame Your JTBD Research with Precision

Build a Robust Job Map for Quantitative Analysis

Design an Outcome-Based JTBD Survey (Quantitative)

Analyze and Visualize JTBD Survey Data

Create a JTBD-Validated Product & Experience Strategy

Apply PCA + JTBD to Advanced Business Decisions

Innovation Leaders seeking to identify underserved markets and high-impact opportunities that shape product and experience strategy.

Product managers looking to prioritize jobs/outcomes using quantitative customer needs data, not just gut instinct.

User and UX Researchers ready to move beyond interviews and turn JTBD insights into statistically valid, data-backed findings.