Change & Measure: Before You Launch Your AB Test
4.6 (9)
5 Days
·Cohort-based Course
Weak A/B test design can hide great ideas beneath insignificant results. Learn 10 checks to avoid throwing away valuable results.
Change & Measure: Before You Launch Your AB Test
4.6 (9)
5 Days
·Cohort-based Course
Weak A/B test design can hide great ideas beneath insignificant results. Learn 10 checks to avoid throwing away valuable results.
Worked with, learned from and/or coached
Course overview
Design better experiments with fewer insignificant & throwaway results
10 test design checks to improve the quality of your experiments:
❌ Loosely set triggers dilute your test, extending its duration and causing a loss of power.
✔️ Learn to set better test triggers.
❌ Underpowered tests are likely to lead to insignificant results.
✔️ Learn to estimate and set realistic Minimum Detectable Effects using historical data.
❌ Compromise and confounding often washes out effects.
✔️ Learn to separate and structure tests with minimum confounding.
❌ Lack of iteration and giving up on an a/b test too quickly leads to wasted effort.
✔️ Learn to design your a/b tests in ways that enables you to iterate.
❌ Lack of supporting metrics limits your ability to make correct decisions.
✔️ Learn to include custom metrics in your a/b tests that reinforce your hypothesis.
❌ Lack of segmenting may hide the effect in the general sample.
✔️ Learn to setup your tests with custom segments that may show effects in smaller subsets of users.
❌ Some changes may be too weak for you to see any difference.
✔️ Learn to setup your tests with different intensities.
❌ Some ideas may be better than others.
✔️ Learn to explore and select ideas from historical data to increase your odds of success.
❌ The effect may be too small from a single change.
✔️ Learn to design compound experiments that combine multiple changes with positive probabilities.
❌ Not all experiments need to test for superiority which demands more power.
✔️ Learn to design non-inferiority tests in some situations.
Who this course is for
01
You're a UI, UX or CRO Test Designer who is frustrated with too many insignificant experiment results.
02
You're an Experimentation & Optimization Strategist responsible for leading A/B testing programs your team and your clients can trust.
03
You're a CRO, Growth or Ecommerce Manager responsible for increasing metrics such as revenue and sales with impactful A/B test results.
By the end of this 1 week course, you'll:
Learn to check your A/B test designs to minimize the odds of insignificant and false positive results
Design A/B tests with more sensitivity
Have fewer throw-away test results
Speed up your experiments
Receive a downloadable template with the A/B test checklist you can use on your projects
Learn from 10-20 common A/B test design scenarios and setups
Join an optional A/B test review session where we'll discuss actively designed experiments and apply the learnings
This course includes
3 interactive live sessions
Lifetime access to course materials
4 in-depth lessons
Direct access to instructor
1 projects to apply learning
Guided feedback & reflection
Private community of peers
Course certificate upon completion
Maven Satisfaction Guarantee
This course is backed by Maven’s guarantee. You can receive a full refund within 14 days after the course ends, provided you meet the completion criteria in our refund policy.
Change & Measure: Before You Launch Your AB Test
Course syllabus
Week 1
May
27
Session 1
Tue 5/273:00 PM—5:00 PM (UTC)May
29
Session 2
Thu 5/293:00 PM—5:00 PM (UTC)May
30
Group A/B Test Reviews (Optional)
Fri 5/303:00 PM—4:30 PM (UTC)Optional
Course Day 1
Course Day 2
Course Day 3: Your A/B Test Reviewed
Bonus
Bonus Resources & Tools
4.6 (9 ratings)
What students are saying
What people are saying
Daria Kurchinskaia
Ron (Ronny) Kohavi
Georgi Georgiev
Harish Venkatesan
.jpeg&w=1536&q=75)
Clemens Grave
Craig Kistler
Meet your instructor
Jakub Linowski
Jakub Linowski has been practicing UI Design (since ~2000) & designing experiments (since 2014). He ❤️ inspiring successful replications by growth and experimentation teams.
He also founded GoodUI.org as an active repository of experiments (with winning, flat and losing results) to separate the more generalizable UI, interaction and copy patterns from the more noisy and less predictable ones.
Over the years, Jakub has either been hired, interacted with, shared results, coached and/or learned from over 100 experimentation teams, including: Microsoft, Thomasnet, Booking, Backstage, Yummly, Elevate, Designlab, Reverb, VivaReal, ZAP, Volders, Rollbar, Kenhub, Expert Institute, Drip Agency, FINN, 3Dhub, Metro to name a few.
Jakub is also known to pay close attention to leaders such as: Netflix, Amazon, Airbnb, Bol, Booking, Etsy and Walmart by simply observing what UI changes they A/B test and then writing about some of their publicly visible experiments.
Jakub holds two design degrees, including a MSc in Design for Interaction (Delft, Netherlands).
Join an upcoming cohort
Change & Measure: Before You Launch Your AB Test
2025-MAY
$795
Dates
Payment Deadline
Course schedule
4-6 hours per week
Tuesdays & Thursdays
11:00am - 1:00pm EST
Friday: A/B Test Reviews
2 hours per week
Learn from real A/B test design scenarios
Course Slide Preview
Have a look at 15 random slides from the course to get a taste for some of the scenarios, test design topics, heuristics (rules of thumb), as well as test design examples.
Preview Slides
Learning is better with cohorts
Active hands-on learning
This course builds on live workshops and hands-on projects
Interactive and project-based
You’ll be interacting with other learners through breakout rooms and project teams
Learn with a cohort of peers
Join a community of like-minded people who want to learn and grow alongside you
Frequently Asked Questions
Stay in the loop
Sign up to be the first to know about course updates.
Join an upcoming cohort
Change & Measure: Before You Launch Your AB Test
2025-MAY
$795
Dates
Payment Deadline