Advanced AB Testing for Data Scientists

New
·

2 Days

·

Cohort-based Course

Master advanced A/B testing to drive data-informed product decisions and measurable growth.

Previously at

Google
Axtria
University of Cincinnati
Walmart
MIT

Course overview

Learn to Design, Analyze, and Scale A/B Tests that Drive Growth

This weekend-intensive workshop is built for data scientists who want to go beyond just “running experiments” — and start shaping product strategy through evidence-based decision-making.


Over two immersive days, you’ll gain hands-on experience in designing robust experiments, analyzing results with rigor, and translating findings into measurable business impact. Through live lectures, breakout discussions, and case-based exercises, you’ll learn not just how to run experiments, but why each design choice matters — from hypothesis framing to metric selection and trade-offs.


This program is fast-paced and practical, tailored for mid-level professionals who want to level up their experimentation craft and drive product outcomes with clarity and confidence.


In this workshop, you’ll cover:

1) Advanced experimental design — from hypothesis framing to metric definition and power analysis

2) Real-world pitfalls — peeking, selection bias, Simpson’s paradox, and multiple testing

3) Statistical analysis techniques — t-tests, non-parametric tests, and confidence interval interpretation

4) Advanced methods — sequential testing, variance reduction, multi-arm bandits, and Bayesian A/B testing

5) Communication and impact — translating results into actionable product and business insights

Who is this course for

01

Data & Applied Scientists who are looking to design stronger experiments and influence product decisions.

02

Data Analysts who want to level up from dashboards to designing and interpreting high-impact experiments.

03

CRO Experts who are optimization professionals seeking deeper statistical rigor and better test design.

What you’ll get out of this course

Design Robust Experiments from Scratch

Learn to define hypotheses, select the right metrics, estimate sample sizes, and randomize correctly to ensure valid and interpretable outcomes.


Identify and Avoid Common Pitfalls

Understand and mitigate challenges like early stopping, Simpson’s paradox, multiple testing, and hidden biases.

Evaluate Test Results Rigorously

Use lift, confidence intervals, and directional insights — not just p-values — to make sound decisions under uncertainty.

Drive Business Impact Through Experimentation

Translate statistical outcomes into actionable insights and foster a culture of data-driven experimentation within your team.

What’s included

Live sessions

Learn directly from Manisha Arora & Banani Mohapatra in a real-time, interactive format.

Lifetime access

Go back to course content and recordings whenever you need to.

Community of peers

Stay accountable and share insights with like-minded professionals.

Certificate of completion

Share your new skills with your employer or on LinkedIn.

Maven Guarantee

This course is backed by the Maven Guarantee. Students are eligible for a full refund up until the halfway point of the course.

Course syllabus

Week 1

Feb 7—Feb 8

    Module 1: Foundations of AB Testing

    3 items

    Module 2: Experimental Design

    4 items

    Breakout Exercise

    1 item

    Module 3: Navigating Pitfalls

    4 items

    Module 4: Analyzing A/B Test Results

    4 items

    Breakout Exercise

    1 item

    Module 5: Advanced Topics

    4 items

    Module 6: Driving Impact

    3 items

Bonus

    PrepVector A/B Testing checklist

    1 item

    Python Code Templates for Experiment Analysis

    1 item

    Curated reading list: papers, blog posts (e.g., Booking.com, Etsy, Airbnb)

    1 item

What people are saying

        I recently took the A/B Testing for Data Scientists & Product Managers course led by Manisha and Banani. The content was well-structured with clear frameworks and engaging case studies that made it easy to apply concepts. I especially valued the introduction to advanced topics like heterogeneity in treatment effects which go beyond the basics.
Soumya

Soumya

Web Analytics and Experimentation Analyst
        This course is an excellent starting point for anyone looking to learn Experimentation. The real-world examples and case studies made the concepts much clearer.
Anargha Ajoy

Anargha Ajoy

Data Scientist, Student
        Manisha's mentorship combined with well structured program and active discussions of practical case studies, played a significant part in elevating my approach and delivery of data science projects. She fostered a supportive, safe and inclusive environment that elevated the quality of discussions. It is a great course to level up your DS skills.
Indu Seetharaman

Indu Seetharaman

Data Scientist, Frost Bank
        I have learnt more from Manisha through her courses than I have learnt in my 4-year college degree. I wish I had found her earlier.
Abhigna Pebbati

Abhigna Pebbati

Analytics & Data Science Manager, Meta
        It was a great deep dive into the fundamentals of AB testing at a level that I couldn't imagine.
Siddarth USC

Siddarth USC

Meet your instructor

Manisha Arora

Manisha Arora

I am a seasoned Data Science professional with 12+ years of experience leading data science teams and driving business growth through data-driven decision making.


I currently lead the Data Science team at Google Ads, working closely with some of the largest advertisers to drive direct $$ to Google.


I am passionate about democratizing data science and enabling others level up in their careers. I found PrepVector to enable aspiring professionals to excel in there data science careers. I have taught 350+ data professionals through my courses at Maven & PrepVector.

Banani Mohapatra

Banani Mohapatra

I am a Data Science Product Leader with 13+ years of experience in e-commerce, payments, and real estate, specializing in transforming data into actionable insights. I am passionate about teamwork and collaboration, I excel at bringing cross-functional teams together to deliver impactful results.


A pattern of wavy dots

Join an upcoming cohort

Advanced AB Testing for Data Scientists

Cohort 1

$1,200

Dates

Feb 7—8, 2026

Payment Deadline

Feb 6, 2026
Get reimbursed

Course schedule

 4 hours Sat, 4 hours Sunday

  • Day 1 – Foundations & Experimental Design

    11:00am - 3:00pm EST

    Build a solid foundation in experimental design. Learn to frame clear hypotheses, define success metrics, estimate sample sizes, and set up robust randomization. You’ll also uncover common pitfalls—like peeking, bias, and confounders—and how to avoid them.

  • Day 2 – Analysis, Interpretation & Advanced Topics

    11:00am - 3:00pm EST

    Go beyond surface-level analysis. Learn to interpret A/B test results with statistical rigor, handle noisy data, & apply advanced methods such as variance reduction, sequential testing, & Bayesian approaches. End the day by connecting experimental insights to real product impact

Learning is better with cohorts

Learning is better with cohorts

Active hands-on learning

This course builds on live workshops and hands-on projects

Interactive and project-based

You’ll be interacting with other learners through breakout rooms and project teams

Learn with a cohort of peers

Join a community of like-minded people who want to learn and grow alongside you

Frequently Asked Questions

A pattern of wavy dots

Join an upcoming cohort

Advanced AB Testing for Data Scientists

Cohort 1

$1,200

Dates

Feb 7—8, 2026

Payment Deadline

Feb 6, 2026
Get reimbursed

$1,200

USD

2 Days