Advanced AB Testing for Data Scientists
Manisha Arora
Data Science Lead, Google
Banani Mohapatra
Data Science Manager, Walmart
.jpg&w=768&q=75)
Learn to Design, Analyze, and Scale A/B Tests that Drive Growth
This weekend-intensive workshop is built for data scientists who want to go beyond just “running experiments” — and start shaping product strategy through evidence-based decision-making.
Over two immersive days, you’ll gain hands-on experience in designing robust experiments, analyzing results with rigor, and translating findings into measurable business impact. Through live lectures, breakout discussions, and case-based exercises, you’ll learn not just how to run experiments, but why each design choice matters — from hypothesis framing to metric selection and trade-offs.
This program is fast-paced and practical, tailored for mid-level professionals who want to level up their experimentation craft and drive product outcomes with clarity and confidence.
What you’ll learn
Master advanced A/B testing to drive data-informed product decisions and measurable growth.
Build hypotheses grounded in product intent and measurable behavior.
Define primary, secondary, and guardrail metrics for decision-making.
Perform power and sample-size analysis to ensure test quality.
Spot peeking, selection bias, Simpson’s paradox, and other pitfalls.
Apply mitigation techniques like clean bucketing and pre-registration.
Diagnose flawed experiments through practical case reviews.
Apply t-tests, non-parametric tests, and alternative methods.
Interpret confidence intervals, effect sizes, and uncertainty.
Compare methods across varying data types and distributions.
Use sequential testing frameworks safely.
Apply variance reduction techniques like CUPED.
Understand multi-arm bandits and Bayesian A/B testing.
Build concise decision summaries highlighting insights and risks.
Convert statistical results into product-focused narratives.
Use storytelling to influence PMs, engineers, and leadership.
.jpg&w=384&q=75)
Who this course is for
Data & Applied Scientists who are looking to design stronger experiments and influence product decisions.
Data Analysts who want to level up from dashboards to designing and interpreting high-impact experiments.
CRO Experts who are optimization professionals seeking deeper statistical rigor and better test design.
Prerequisites
Basic Understanding of AB Testing
You should be familiar with the basics of experiment design. Check out AB Testing 101 course here
What's included
Live sessions
Learn directly from Manisha Arora & Banani Mohapatra in a real-time, interactive format.
Lifetime access
Go back to course content and recordings whenever you need to.
Community of peers
Stay accountable and share insights with like-minded professionals.
Certificate of completion
Share your new skills with your employer or on LinkedIn.
Maven Guarantee
This course is backed by the Maven Guarantee. Students are eligible for a full refund up until the halfway point of the course.
Course syllabus
Week 1
Module 1: Foundations of AB Testing
Module 2: Experimental Design
Breakout Exercise
Module 3: Navigating Pitfalls
Module 4: Analyzing A/B Test Results
Breakout Exercise
Module 5: Advanced Topics
Module 6: Driving Impact
Bonus
PrepVector A/B Testing checklist
Python Code Templates for Experiment Analysis
Curated reading list: papers, blog posts (e.g., Booking.com, Etsy, Airbnb)
Free resource
Unlock Data Science Job Opportunities
Understand Key Data Roles in the Industry
Gain insights into trends and demands in the job market. I will break down responsibilities and skills for each role
Assess Your Strengths and Growth Areas
We will perform a quick assessment of your skills to identify strengths and growth areas so you can find suitable roles
Find a Role for Long-term Success
Find a role that best fits YOUR skills and passions, to bring joy and long-term success
Interview Preparation Tips
Demonstrate your ability to solve complex problems through data-driven insights and technical excellence
Schedule
Live sessions
4-6 hrs / week
Async content
2-4 hrs / week
Testimonials
- I recently took the A/B Testing for Data Scientists & Product Managers course led by Manisha and Banani. The content was well-structured with clear frameworks and engaging case studies that made it easy to apply concepts. I especially valued the introduction to advanced topics like heterogeneity in treatment effects which go beyond the basics.
Soumya
Web Analytics and Experimentation Analyst - This course is an excellent starting point for anyone looking to learn Experimentation. The real-world examples and case studies made the concepts much clearer.
Anargha Ajoy
Data Scientist, Student - Manisha's mentorship combined with well structured program and active discussions of practical case studies, played a significant part in elevating my approach and delivery of data science projects. She fostered a supportive, safe and inclusive environment that elevated the quality of discussions. It is a great course to level up your DS skills.
Indu Seetharaman
Data Scientist, Frost Bank - I have learnt more from Manisha through her courses than I have learnt in my 4-year college degree. I wish I had found her earlier.
Abhigna Pebbati
Analytics & Data Science Manager, Meta - It was a great deep dive into the fundamentals of AB testing at a level that I couldn't imagine.
Siddarth USC
Director of Data Science, Microsoft
Frequently asked questions
$1,200
USD