Jakub Linowski

JAN-2025

DEC-2024

ClearlinkJan2025

2025-july

APR2025

2025-MAY

2025 December

Have a look at 15 random slides from the course to get a taste for some of the scenarios, test design topics, heuristics (rules of thumb), as well as test design examples.

Course Slide Preview

Get a preview of a 5 leap experiment examples we'll discuss in more depth in the course. This small preview includes experiments I observed ran by: Amazon, Walmart and Netflix as well as one experiment I designed for Designlab.

See 5 Leap Experiment Examples From Leading Companies

Receive 5 example leap experiments from leading companies, including: Amazon, Walmart, Netflix and Designlab.

5 Leap Experiments (Netflix, Walmart, Amazon)

Weak A/B test design can hide great ideas beneath insignificant results. Learn 10 checks to avoid throwing away valuable results.

Change & Measure: 10 Checks For Designing Better AB Tests

Sr. Director Personalization & Experimentation

Analytics Consultant

Product Designer 

Product Lead

Design Ops

Optimisation Lead

Head of eMerchandising

Founder

Director, Data Science

Director, Marketing Optimization

10 test design checks to improve the quality of your experiments: ❌ Loosely set triggers dilute your test, extending its duration and causing a loss of power.✔️ Learn to set better test triggers. ❌ Underpowered tests are likely to lead to insignificant results.✔️ Learn to estimate and set realistic Minimum Detectable Effects using historical data. ❌ Compromise and confounding often washes out effects.✔️ Learn to separate and structure tests with minimum confounding. ❌ Lack of iteration and giving up on an a/b test too quickly leads to wasted effort.✔️ Learn to design your a/b tests in ways that enables you to iterate. ❌ Lack of supporting metrics limits your ability to make correct decisions.✔️ Learn to include custom metrics in your a/b tests that reinforce your hypothesis. ❌ Lack of segmenting may hide the effect in the general sample.✔️ Learn to setup your tests with custom segments that may show effects in smaller subsets of users. ❌ Some changes may be too weak for you to see any difference.✔️ Learn to setup your tests with different intensities. ❌ Some ideas may be better than others.✔️ Learn to explore and select ideas from historical data to increase your odds of success. ❌ The effect may be too small from a single change.✔️ Learn to design compound experiments that combine multiple changes with positive probabilities. ❌ Not all experiments need to test for superiority which demands more power.✔️ Learn to design non-inferiority tests in some situations.

Design better experiments with fewer insignificant & throwaway results

Course syllabus

Tuesdays & Thursdays

Friday: A/B Test Reviews

Course schedule

You're a UI, UX or CRO Test Designer who is frustrated with too many insignificant experiment results.

You're an Experimentation &amp; Optimization Strategist responsible for leading A/B testing programs your team and your clients can trust.

You're a CRO, Growth or Ecommerce Manager responsible for increasing metrics such as revenue and sales with impactful A/B test results.

Who this course is for

By the end of this 1 week course, you'll:

Learn to check your A/B test designs to minimize the odds of insignificant and false positive results

Design A/B tests with more sensitivity

Have fewer throw-away test results

Speed up your experiments

Receive a downloadable template with the A/B test checklist you can use on your projects

Learn from 10-20 common A/B test design scenarios and setups

Join an optional A/B test review session where we'll discuss actively designed experiments and apply the learnings

UI Experiments From Leading Companies

Experimentation & Design Leader • GoodUI Founder | Coached ~100 Growth Teams

I'll share selected UI experiments ran and iterated on by: Netflix, Airbnb, Amazon, Walmart and others.

See how leading companies have designed experiments

Understand that not all experiments are the same. See how companies run different types of experiments.

See a variety of experiment designs

Types Of Experiments Rundown

Experiment types: Isolated, Intensity, Leap, Exploratory, Cascade, Cluster Randomized, Replication, Switchback, Hold Out

Widen your experiment design horizons with an overview of:

We'll also discuss how different experiments types could be combined together to complement their strengths.

Think of experiments as sequences

It's a live cohort, so we'll make room for your input on interesting experiments types & combinations.

Learn directly from Jakub Linowski