Mindy Yang

Why Chocolate Tastes Sweeter with Music

Sipsmith Gin

Veuve Clicquot

Harney & Sons Fine Teas

Tequila Herradura

Glenmorangie

Co-Founder & CEO of World Taste & Smell Association, CEO of Perfumarie

Recognize the role of cross-modal cues (sound, shape, texture) in altering chocolate’s taste.

Experience how sensory design influences emotion and storytelling in hospitality and marketing

Gain a preview of the tools and insights offered in the 3-part masterclass

Through multiple real examples of well-run experiments and real stories at Microsoft, Amazon, and Airbnb, you will see the humbling reality that we are terrible at assessing the values of ideas. Trivial changes can be surprisingly useful, whereas large efforts often fail. Accelerate innovation by building Minimum Viable Products and Features (MVPs) and make the organization evidence-based and humbler, as it adopts and learns to use evidence from the gold standard in science: the controlled experiment. You will understand the challenges in designing and running trustworthy controlled experiments, or A/B tests, including the importance of the Overall Evaluation Criterion (OEC), scaling, pitfalls, and Twyman's law. The 2nd week covers additional topics, some more technical, including cultural challenges, institutional memory, maturity model, observational causal studies, offline evaluations, AI/Machine learning and triggering, scaling, build vs. buy, challenges, and requested topics. The advanced topics course is a follow-on to this: https://bit.ly/AdvancedABFS

Empower your organization to be data-driven and innovative

Course syllabus

Three x 2-hour sessions + 1/2-hour bonus in week 1 on Monday, Tuesday, and Thursday.The bonus time is optional. Material may be more technical, and time for additional Q&amp;A

Week 1: Mon, Tue, Thu

Two x 2-hour sessions + 1/2-hour bonus in week 2 on Monday and Thursday

Week 2: Mon, Thu

Optional Q&A

Course schedule

Data science managers and scientists will be able to design and interpret the experiment results in a trustworthy manner

Program managers focused on growth, revenue, conversions, and prioritization will understand how to provide the org with robust clear metric

Engineering leaders will be able to make the organizations more data-driven and efficient with fewer severe incidents through A/B tests

Who is this class for?

What you will learn from this 10-hour course

You will hear multiple real memorable stories and examples, many that don't make it to books or articles. These were chosen from over 20 years of experimentation. You'll have the data to show that the poor success rate is documented across multiple organizations; expected it!

Understand and internalize the humbling reality that we are poor at assessing the values of ideas: most ideas fail!

Understand key concepts like causality, hierarchy of evidence, and key organizational tenets required for effective experimentation.

Understand the key advantages and limitations of A/B testing

Designing metrics is hard. There is a hierarchy of metrics and perverse incentives. The most important metrics comprise of the OEC - The Overall Evaluation Criterion. We will look at good and bad examples.

Learn how to design metrics and the Overall Evaluation Criterion

Getting numbers is easy; getting numbers you can trust is hard. You'll understand common pitfalls and how to design reliable and trustworthy tests.

Learn how to designing trustworthy A/B tests

Learn about the cultural challenges, the humbling results (most ideas fail, pivoting, iterating, learning), institutional memory, ideation, prioritization, experimentation platforms

Learn about the cultural challenges

When building AI or machine learning models, using A/B testing and triggering to evaluate the models that were built offline based on historical data

Learn about the relationship to AI and Machine Learning Modeling

When you can't run an A/B test, quasi-experimentation methods and the risks of observational causal studies

Learn about complementary quasi-experimental techniques

What are key challenges and open questions in the field

Learn about existing challenges

If there is something specific you want to cover, there is time allocated for topics voted by the audience to discuss

Learn about YOUR topic of interest

The course focuses on developing the intuition and common misunderstandings, without the details of the statistics, which you can find in many books. We cover p-values, statistical power, and triggering.You can go as technical as you want in the Q&amp;A and community discussions

Technical to your needs

Learn from a world-leading expert how to design and analyze trustworthy A/B tests to evaluate ideas, integrate AI/ML, and grow your business

Microsoft

Airbnb

Amazon

Dr. Ronny Kohavi

Executive at Microsoft, Airbnb, Amazon, author of best-selling A/B testing book

8 Sept 2025

1 Dec 2025

This weekend workshop is a hands-on introduction to A/B Testing designed specifically for data scientists and product managers. To truly drive impact, you must connect experiment results to business decisions, collaborate effectively with product stakeholders, and communicate findings in ways that influence roadmaps. This workshop is designed to help mid-senior data professionals build exactly those skills, sharpening your product thinking and transforming how you approach experimentation. This course offers a practical, fast-paced dive into real-world experimentation. Through interactive case-studies, group discussions, and live walkthroughs of industry tests, you will learn how to:1) Frame hypotheses that lead to actionable decisions, not just statistical tests2) Choose metrics that reflect business trade-offs — primary, secondary, and guardrails3) Interpret test results in business terms, beyond just “stat sig”4) Communicate insights clearly to PMs, leadership, and cross-functional teams By the end, you won’t just be running A/B tests — you’ll be driving better product decisions with them. We will wrap up the discussion by learning about what separates a good data scientist from a great one, and how you can continue building on your data skills.

Learn to Design and Analyze A/B Tests that Influence Product Strategy.

Learn how great experiments start—with the right questions. We’ll cover hypothesis framing, choosing the right metrics, and designing experiments that align with product goals. You’ll also explore real-world examples of good (and not-so-good) test design.

Day 1: Foundations of Experimentation

We’ll dive into analyzing experiment outcomes, understanding tradeoffs, and handling ambiguous results. You’ll practice translating test results into product impact and learn how to communicate insights to influence decision-making.

Day 2: Influence Decision-Making and Drive Outcomes

Data scientists who want to build A/B Testing skills to share business outcomes and get interview-ready.

Growth analysts looking to design high-impact experiments and optimize key funnel metrics.

Product Managers who work closely with data teams and want to better frame, interpret and act on experiment results.

Who this course is for:

What you’ll get out of this course

Give students an idea of how they can expect to grow throughout your course. Include specificity and precise results so students can benchmark exactly what they’ll learn.

Design high-signal A/B tests by framing clear hypotheses and selecting the right success and guardrail metrics aligned with product goals.

Evaluate the impact of test results using confidence intervals, lift, and directional trade-offs to recommend feature launch decisions.

Analyze real-world case studies from Uber and Spotify to learn the applications of AB Tests to industry challenges.

Create a clear decision brief that translates experiment results into business-relevant insights for PMs, engineers, and leadership.

Learn a practical framework to evaluate real-world A/B tests, identify weak hypotheses, misaligned metrics, and missed opportunities based on product intent.

Apply a reusable A/B testing framework to critique and improve existing experiment setups based on product context and goals.

Learn to design A/B tests that drive real product decisions - from metrics and hypotheses to business impact.

Manisha Arora

Data Science Lead, Google | Founder, PrepVector | MIT, UT Austin & Univ of Cinc

Banani Mohapatra

Senior Manager, AI/ML & Data Science at Walmart | Generative AI,LLM | IIT Delhi

Cohort 1

Cohort 2

This 8-week cohort helps you build a brand that doesn’t just sell, but resonates—leveraging the BELONG™ Framework to ground your marketing in applied psychology, human connection, and ethical influence. With 80% of consumers more likely to purchase from brands that offer personalized experiences (Epsilon) and 90% of millennials saying authenticity is important when choosing who to support (Stackla), the need for human-centered marketing has never been greater. You’ll learn how to harness behavioral science and neuroscience to connect with how people actually think and feel while using AI tools to amplify your humanity, scale your impact, and automate with authenticity. By the end of the cohort, you'll be able to view marketing through a new lens (a psychologically aligned one) AND you’ll have a complete, psychologically aligned marketing plan ready to implement. You’ll Walk Away With:- A crystal-clear understanding of your ideal audience and what drives their decisions- A Value Aspiration Statement that aligns your mission with your customer’s deeper needs- Messaging that’s rooted in neuroscience and designed to engage attention and emotion- A full, ethically aligned funnel that attracts and retains the right people organically- A complete marketing plan tailored to your business goals and customer journey How We’ll Do It:- Weekly one-hour live trainings that move from insight to implementation through the BELONG™ framework- Real-world examples, scripts, and messaging rewrites that show you how, not just what- Biweekly office hours with hot seats to work through blockers (optional) and refine your work in real time- A bonus 1:1 implementation session to finalize your plan and get expert feedback

Build Your Ethical Brand and Acheive Sustainable Growth

Wednesdays are 60-minute live sessions focused on structured teaching, real-world examples, and Q&amp;A. Tuesdays are biweekly 60-minute office hours where you can join hot seats, move through blockers, and get personalized support from Chelsea and your peers.

Tuesdays & Wednesdays

Each week focuses on a core element of the BELONG™ Framework with clear, actionable assignments. You’ll receive written recaps, templates, and examples to guide your implementation and build your final marketing plan step by step.

Weekly Projects

Every participant receives a bonus 1:1 session with Chelsea at the end of the cohort. We’ll review your plan, identify next steps, and make sure you’re set up to implement with confidence and clarity.

Private 1:1 Implementation Session with Chelsea

Small Business OwnersStill wearing all the hats? Make sure your marketing aligns with your values — and actually works.

EntrepreneursWhether you're just starting or scaling, build a marketing approach that honors your mission and values.

In-House MarketersUplevel your strategy with psychology-backed tools that foster belonging and real customer connection.

Who is this course for

Knowing your audience is about more than demographics. We’ll use psychology to uncover how they see themselves, what motivates them, and what actually drives their decisions. You’ll leave with an identity map and decision drivers that shape every part of your marketing.

Identify exactly who your audience is—and what drives them

You don’t have to choose between your mission and your marketing. We’ll identify the overlap between your business’s higher purpose and your customer’s deeper aspirations. This is the foundation of ethical marketing that resonates—and converts.

Align your values with what your audience truly wants

Want your marketing to stick? We’ll explore how attention, memory, and motivation work—and how to design messaging that taps into each. You’ll practice rewriting your own content using psychology-backed frameworks that build trust and emotional resonance.

Craft messaging that works with the brain, not against it

You don’t need to hustle or rely on ads to build consistent interest. We’ll map out attraction, engagement, and retention loops that create organic, ongoing growth. You’ll leave with a sustainable system you can actually maintain.

Design a sustainable demand system that grows with you

Your brand story isn’t about you—it’s about who your audience becomes through you. We’ll use storytelling psychology to help you write from their point of view and create connection. You’ll get frameworks, feedback, and a ready-to-use brand narrative.

Tell a story your audience sees themselves in

Marketing isn’t one message—it’s a journey. We’ll map your customer’s decision-making process and align your strategy at every stage. You’ll finish with a full psychological journey map that turns attention into action and trust into loyalty.

Guide customers from first touch to full transformation

Each week, you’ll create part of your marketing strategy — from audience insights to messaging, funnels, and brand story. By the end of the cohort, you’ll bring it all together into a full, cohesive marketing plan that reflects your values and speaks directly to your audience.

Build a complete, psychologically aligned marketing plan you can actually use

Your bonus implementation call is designed to turn clarity into action. We’ll review your full plan together, identify gaps or sticking points, and outline your next steps—so you walk away with not just ideas, but confidence in how to move forward.

Get expert feedback in a 1:1 implementation session to bring your plan to life

Your marketing can be authentic, powerful, AND profitable. Build an ethical brand with a full marketing plan all based on psychology.

Build an Ethical Brand Cohort 2

Chelsea Burns

Build an Ethical Brand Cohort 1

Build an Ethical Brand Cohort 3

But most teams are still designing like it’s 2015. AI is transforming industries and changing the very foundation of how products work. Companies are moving fast to integrate large language models, agents, and adaptive systems into their products. But as model complexity grows, the user experience often gets left behind. Powerful algorithms can now predict, generate, and decide but without thoughtful design, they often confuse users, introduce risk, or fail to earn trust. The challenge? Designing for AI isn’t like designing for traditional software. It’s not just about usability or polish anymore. It’s about system behavior, adaptation, and collaboration between humans and machines. If you’ve ever launched an AI feature only to see users abandon it because they didn’t understand what it was doing or if you’ve been asked to “design the AI experience” but weren’t sure where to begin → this course is for you. The Role of the Designer Is ChangingUntil recently, designers focused on interfaces and flows. Today, the frontier is different. OpenAI recently posted a job for a Model Designer, a new kind of design role focused on shaping how models behave. And it signals a broader shift across the industry: Designers aren’t just styling outputs anymore. They’re architecting how intelligent systems learn, adapt, and interact. In AI-powered products, design happens upstream, at the level of:- Data inputs and how users shape them- Model decisions and how they’re surfaced or corrected- Feedback loops that guide learning- System behavior that evolves over time To adapt to this new environment, designers must understand how models work, not to become ML engineers, but to ensure what the system learns, shows, or hides aligns with human expectations and values. Real Challenges This Course Prepares You For You launch an AI-powered feature and maybe it recommends content, summarizes input, or automates a task. But users are confused or drop off quickly. They can’t tell why the AI did what it did, or how to influence it. The model works, technically. The experience doesn’t.→ Learn to design transparency, feedback, and control into intelligent systems before trust is broken. Your CTO asks you to “design the AI experience.” There’s no interface yet, just a prototype from the ML team and a prompt to “make it usable.” You’re left staring at system outputs, unclear where to start, or how much of the system behavior is even within your control.→ Gain the tools and frameworks to work upstream, shaping behavior, not just UI. You’re in a meeting where the team is debating whether to launch a model with 72% accuracy. Product wants speed. ML is confident the model will improve post-launch. But no one’s talking about the UX cost of being wrong nearly 1 in 3 times or how the system will recover from those mistakes.→ Learn how to design for failure, uncertainty, and system correction from day one. A personalization feature goes live. But user feedback is harsh: “This feels random.” You realize the issue wasn’t the model, it was the lack of communication and guidance. There’s no explanation, no context, and no way for users to improve the output.→ Understand how to design model feedback loops and trust-building interactions. You inherit a half-built AI assistant. It occasionally interrupts users at the wrong time, answers inconsistently, and disappears when needed most. The UI looks fine but the behavior feels off. You realize this isn’t a layout problem, it’s a systems design problem.→ Move from UI design to shaping system behavior, flows, and agent logic. You will bring your own real-life project scenarios, and we’ll coach you through how to solve them—using frameworks, feedback, and hands-on critiques.

The AI Tech Revolution Is Redefining Design

Class will meet once a week for live lectures and project discussions.

Weekly Live Sessions

Plan to work on a capstone project to apply your learning to a capstone project

Weekly assignments

Get weekly 1-1 office hours for personalized feedback on your capstone project

1-1 Coaching 

Senior UX &amp; Product Designers: Deepen your systems thinking skills and learn to design around model behavior

Design Leads &amp; Heads of UX: Guide teams, shape AI product strategy, and lead with system-level thinking and trust design.

AI-Aware PMs &amp; Researchers: Collaborate better with design, align model goals with UX, and support ethical, adaptive systems.

Learn how to speak the language of models, data pipelines, and evaluation metrics—so you can contribute meaningfully from early scoping to final delivery. You’ll practice framing use cases, defining handoffs, and aligning design with system capabilities.

Collaborate effectively with ML and Product teams

Instead of just designing around the model, you’ll explore how interaction design can shape what the model learns, retains, or ignores. You’ll build UX patterns that create useful training signals, structured inputs, and adaptive guidance.

Shape model behavior through smart UX architecture

You’ll create flows that help users understand, question, or override the model’s decisions—like confidence indicators, correction mechanisms, and recovery paths for failure scenarios.

Design interactions that support user trust, control, and adaptation

Learn to design feedback loops where user actions (like corrections, inputs, skips, or feedback) help the model adapt. You’ll map those flows and create testable patterns that help close the loop between model performance and user value.

Build systems that improve over time—with users in the loop

Whether you’re dealing with ranking systems, generation models, or multi-step agents—you’ll practice mapping decision logic, designing trust dashboards, and identifying failure points. You’ll learn how to advocate for UX in model evaluation and behavior design conversations.

Lightning Lessons

Why Chocolate Tastes Sweeter with Music