Through multiple real examples of well-run experiments and real stories at Microsoft, Amazon, and Airbnb, you will see the humbling reality that we are terrible at assessing the values of ideas. Trivial changes can be surprisingly useful, whereas large efforts often fail. Accelerate innovation by building Minimum Viable Products and Features (MVPs) and make the organization evidence-based and humbler, as it adopts and learns to use evidence from the gold standard in science: the controlled experiment. You will understand the challenges in designing and running trustworthy controlled experiments, or A/B tests, including the importance of the Overall Evaluation Criterion (OEC), scaling, pitfalls, and Twyman's law. The 2nd week covers additional topics, some more technical, including cultural challenges, institutional memory, maturity model, observational causal studies, offline evaluations, AI/Machine learning and triggering, scaling, build vs. buy, challenges, and requested topics. The advanced topics course is a follow-on to this: https://bit.ly/AdvancedABFS

Empower your organization to be data-driven and innovative

<ul><li>Introduction to controlled experiments, or A/B testing</li><li>Interesting experiments – you’re the decision maker</li><li>Organizational tenets</li><li>Advantages and limitations of controlled experiments</li></ul>

Introduction

<ul><li>Interesting example</li><li>End-to-end experiment: Running and Analyzing an Experiment</li><li>Metrics</li><li>The OEC</li></ul>

End-to-end example, and Metrics and the OEC

<ul><li>Interesting example</li><li>P-values and statistical power</li><li>End-to-end experiment: speed matters</li><li>Twyman’s Law and Trustworthy Experimentation</li><li>Generating ideas, prioritization, ethics</li></ul>

P-values and power, End-to-end Example 2, Twyman's law and Trustworthy Experimentation

<ul><li>Interesting example</li><li>Cultural challenge, Institutional memory, maturity model</li><li>Observational causal studies and quasi-experimental designs</li><li>Pitfalls in observational studies</li></ul>

Cultural challenges, Prioritization, AI/Machine Learning, Complementary Techniques and Observational Causal Studies

<ul><li>AI/Machine Learning model, and triggering</li><li>Leakage and interference</li><li>Bayesian A/B Testing</li><li>Experimentation Platform and scaling</li><li>Build vs. Buy</li><li>Challenges</li><li>Requested Topics</li></ul>

Advanced topics, challenges, and requested topics

Course syllabus

Three x 2-hour sessions in week 1 on Monday, Tuesday, and Thursday

Week 1: Mon, Tue, Thu

Two x 2-hour sessions in week 2 on Monday and Thursday

Week 2: Mon, Thu

Optional Q&A

Course schedule

Data science managers and scientists will be able to design and interpret the experiment results in a trustworthy manner

Program managers focused on growth, revenue, conversions, and prioritization will understand how to provide the org with robust clear metric

Engineering leaders will be able to make the organizations more data-driven and efficient with fewer severe incidents through A/B tests

Who is this class for?

What you will learn from this 10-hour course

You will hear multiple real memorable stories and examples, many that don't make it to books or articles. These were chosen from over 20 years of experimentation. You'll have the data to show that the poor success rate is documented across multiple organizations; expected it!

Understand and internalize the humbling reality that we are poor at assessing the values of ideas: most ideas fail!

Understand key concepts like causality, hierarchy of evidence, and key organizational tenets required for effective experimentation.

Understand the key advantages and limitations of A/B testing

Designing metrics is hard. There is a hierarchy of metrics and perverse incentives. The most important metrics comprise of the OEC - The Overall Evaluation Criterion. We will look at good and bad examples.

Learn how to design metrics and the Overall Evaluation Criterion

Getting numbers is easy; getting numbers you can trust is hard. You'll understand common pitfalls and how to design reliable and trustworthy tests.

Learn how to designing trustworthy A/B tests

Learn about the cultural challenges, the humbling results (most ideas fail, pivoting, iterating, learning), institutional memory, ideation, prioritization, experimentation platforms

Learn about the cultural challenges

When building AI or machine learning models, using A/B testing and triggering to evaluate the models that were built offline based on historical data

Learn about the relationship to AI and Machine Learning Modeling

When you can't run an A/B test, quasi-experimentation methods and the risks of observational causal studies

Learn about complementary quasi-experimental techniques

What are key challenges and open questions in the field

Learn about existing challenges

If there is something specific you want to cover, there is time allocated for topics voted by the audience to discuss

Learn about YOUR topic of interest

The course focuses on developing the intuition and common misunderstandings, without the details of the statistics, which you can find in many books. We cover p-values, statistical power, and triggering.You can go as technical as you want in the Q&amp;A and community discussions

Technical to your needs

Learn from a world-leading expert how to design and analyze trustworthy A/B tests to evaluate ideas, integrate AI/ML, and grow your business

Dr. Ronny Kohavi

2 Dec 2024

3 March 2025

Take the guesswork out of how to write good messaging. In this course, you’ll learn how to translate your technology into clear and compelling messaging that resonates with your target customer on a deeper level – even if you’re new to messaging. Using frameworks and formulas, you’ll workshop messaging for your own product or company in a small group setting.&nbsp; This cohort-based course includes teaching, workshopping and peer discussion, feedback from Emma, Q&amp;A, and templates and resources.

Build the skills (and confidence) to write good messaging

Why strong messaging starts with customer motivation.&nbsp;You'll learn what customers care about, where to research those things, and how to get real so that your messaging actually resonates.

Lesson 1: Customer pains and gains

The rules of value-focused messaging. You'll learn the difference between features, benefits, and value propositions, and how they affect messaging hierarchy. &nbsp;

Lesson 2: Features, benefits, and value props
 

Speed up and strengthen headline and body copy creation.&nbsp;You'll learn what strong, punchy headlines and body copy look like, why they matter, and how to write them.

Lesson 3: Punchy messaging and copy

A step-by-step process for clear, concise, big-impact writing.&nbsp;You'll learn easy, bang-for-your-buck writing and editing tips, and a proven way to think through your writing process and execute at speed.

Lesson 4: Power writing

Your Punchy training syllabus

<ul><li>On-demand Video Access: Thursday, February 8th</li><li>Live Workshop 1: Thursday, February 22nd @ 9am-12pm PST</li><li>Live Workshop 2: Friday, February 23rd @ 9am-12pm PST</li><li>Live Feedback Session: Thursday, March 7th @ 9-11 am PST</li><li>Q&amp;A Session: Friday, March 8th @ 9-10 am PST</li></ul>

Cohort 3: February 22nd & 23rd, 2024

Next available courses

Product marketers who want to write bolder, more human messaging for their tech products and solutions.

B2B marketing pros who want to write clear, concise (and punchy!) messaging that connects with people.

Tech founders who want to write messaging that makes their product sound simpler and more compelling.

If you write for B2B, this is for you.

What you'll get out of this course

Get to good messaging faster with fewer rewrites and revisions (and less frustration).

Write better messaging faster with frameworks and formulas

Follow frameworks for better brainstorming so you know how to get out of messaging ruts and into new perspectives.

Generate fresh ideas and new angles for your messaging

Get 1-on-1 feedback from Emma and learn from fellow marketers facing similar messaging challenges

Workshop your messaging with peers

Get clear on how to take messaging from good to great every time and realize how awesome you are at this.

Feel confident about your messaging chops

Learn how to write simple, human messaging that resonates with your customer and improves your B2B marketing.

Emma Stratton

Punchy Founder

Cohort 3

Cohort 4

Leveraging community as a growth channel requires a nuanced understanding of human behavior, scalable strategies, and a deep commitment to delivering value. This course will teach you how to build, grow, and scale a thriving community-led growth engine that delivers powerful, long term results: Increased Brand Loyalty: Consumers who feel a sense of belonging to a brand community are more likely to remain loyal to the brand over time. Emotional Connection: A strong community fosters emotional attachment, which can increase retention rates as people are less likely to switch to competitors. Influence of Social Proof: Community-driven consumers are much more likely to recommend the brand to others, reinforcing their own commitment and reducing churn. These recommendations also drive much lower marketing costs and higher conversion rates at the consideration phase. Resilience to Negative Experiences: Consumers embedded in a community are more forgiving, as they perceive a deeper relationship beyond mere transactional value. Why learn with me?I've been driving growth through user behavior for over a decade across multiple industries. I've gained critical insights about how to build user-led growth loops, ecosystem engagement and brand affinity.I've road-tested my strategies across multiple operator roles and organizations, and learned how to define what CLG should look like for each business, based on its unique model and goals.In this course, I'll equip you to do the same, for your specific use case. You'll leave this course with: A thorough understanding of CLG as a concept and strategyA wide variety of examples to pull from as you ideate on your own CLG approachSpecific feedback on your existing CLG plans, ideas, and questions

Supercharge your brand’s growth and retention through natural human behavior.

<ul><li>Overview of community-led growth tactics and strategies</li><li>Case studies of successful CLG implementations across B2B and B2C</li></ul>

Foundations of Community-Led Growth


<ul><li>Driving brand loyalty and community engagement</li><li>Psychological frameworks that prompt desired user actions</li></ul>

Leveraging User Psychology for Growth Loops


<ul><li>Community surfaces and types</li><li>Paid vs free community models</li><li>Software tools to support community growth</li></ul>

Implementation Strategies


<ul><li>We'll meet for 15 min to discuss your current problem set live, then you'll do some asynchronous work which I'll provide feedback on to help you make real progress</li></ul>

Your current problem set


What we'll cover:

Fundamentals of Community-Led Growth<ul><li>Overview of CLG tactics and strategies for user acquisition and retention</li><li>Case studies of successful community-led brands</li><li>Measuring success and results</li></ul>

Monday: Live session


Leveraging Psychology for Growth<ul><li>Tactics for brand loyalty and hype creation</li><li>Psychological frameworks for building engaged, sustainable community loops </li><li>Psychological frameworks for encouraging desired user actions</li></ul>

Tuesday: Live session


<ul><li>A 15 minute slot to discuss specific challenges and problem sets to you and your role, 1:1</li></ul>

Wednesday: Office hours


Implementation Strategies<ul><li>Community surfaces and types</li><li>Paid vs. free community models</li><li>Software tools to support community growth</li></ul>

Friday: Live session


<ul><li>Submit your plan or ideas for how to implement CLG in your work</li><li>Receive async feedback, ideas, and suggestions within a week</li></ul>

Week 2: Personalized feedback


Schedule

a marketer tasked with finding new growth channels and paths beyond paid media

a product builder thinking about creating growth loops and driving PLG

a growth owner who wants to harness to power of word of mouth to lower acquisition costs

This course is for you if you're...


You'll leave this course with:

Well-designed ommunity-led growth looks different for every brand and company. We'll review a variety of well-tested strategies across B2B, B2C, and B2B2C models in order to help you define what it could, and should, look like for yours based on your model and goals.

A clear understanding of how to integrate community-led strategies into user acquisition and retention efforts.

By studying a series of psychological frameworks and triggers, we can design our growth efforts to be multitudes more impactful and effective at driving our desired user actions. We'll dive into the ones most relevant to CLG motions, and analyze when and how to implement them.

Practical insights into leveraging psychological triggers to build loyalty and create buzz around your brand.

Community products are generally loss leaders for larger brands, but someties it does make sense to position them as an upsell or paid feature. We'll discuss the pros and cons of each approach, and how that might apply to the problems you're solving.

Actionable frameworks for deciding between paid and free community models.

Having the right tech stack is critical for CLG impact and success. I'll provide recommendations for tools to drive high quality user experience and the measurement and automation tools you'll need to scale sustainably,

A toolkit of recommended software and resources to scale your community initiatives.
 

Following our live sessions, you'll submit your own proposal for how to implement CLG into your work. I'll provide personalized feedback and suggestions to optimize outcomes and set yourself up for long term success.

Tactical feedback on your specific problem set and proposed solutions.

⚡ Get the tools, frameworks, psychological insights, and tactics to sustainably scale your CLG
👉 Create new, high ROI growth loops

Mallory Contois

5x startup operator @ Linkedin, Pinterest, Cameo, Metafy & Mercury

March cohort

_____________07:00 PM (19:00), GMT+2, five sessions over 2.5 Weeks_____________ The top quartile SaaS companies between 5 and 20m in revenue are growing over 70% year over year, with an 8 month CAC payback, 110% Net dollar retention while having an 85% Gross Margin with an average Annual Revenue of over 250’000$. (Source: Growth Unhinged 2023 Report) And at the heart of most them sits a deep understanding of Product-led Growth. * How do we in B2B adress the growing problem of everyone asking for an improved buying experience and self-serve? * How do you package your offering with freemium, trial and/or interactive demo? * How do we activate individuals, teams and accounts in B2B as cost efficient as possible to drive our pipeline? * How does Product-led Growth work with an existing Sales motion? * How do we structure a strategy to bring product-led growth into our business? Leah Tharin brought product-led growth as an operative, executive, and advisor to dozens of scale up companies to optimize their user acquisition, retention, and monetization. She's recognized as an Industry leader on the topic having written some of the most referenced Product-led Growth &amp; Sales material to date. Learn how to structure a (product-led) growth strategy that works for your business. It is designed for leaders from all functions who want to understand and not just blindly execute. The course runs over 2.5 weeks (5 live recorded Zoom classes). All sessions have prerecorded material for you to study in your own time, the live sessions are solely focussed on world-class guest speakers, collaborative group exercises and your questions. You can expect per week about 2-3 hours of study for the asynchronous material at your own pace + 60-75 minutes per live session. Over 20 Guest speakers from past cohorts for example :* Adam Fishman, SVP new products @ Mozilla, Investor &amp; Advisor* John Cutler, ex- Evangelist @ Amplitude &amp; Toast* Itxaso Del Palacio, General Partner @ Notion VC* Benjamin Lamson, Head of Paperspace Revenue @ DigitalOcean* Casey Hill, Sr. Growth Manager @ Active Campaign* Henrique Cruz, Head of Growth @ Rows.com* Sara Gattoni, Senior Lead Growth PM @ Lokalise* Andrew Davies, CMO @ Paddle...and many more This course is not a good fit for: * Companies pre product market fit which are extremely early in their journey. * Non-software or D2C companies * Highly regulated industries that deal with government contracts

How experienced Growth operators use PLG in B2B SaaS

…the definition of PLG…where PLG sits in the market…when you might do PLG…PLG vs Saled-led Growth…Enterprise vs. Midmarket vs. Micro businesses…Best Conditions

What is product-led Growth

…freemiums…trials…reverse-trials…case studies, examples in the wild

Freemium, trial, reverse-trial

…understanding customer success stages…segmenting your users into ICPs (Ideal customer profiles)…turning funnels into loops, focussing on what matters

Defining customer success & ICPs

…separating the creation from capturing value…smart goal setting company wide

PLG strategy and goal setting

…Classical vs Growth Teams…(Growth) Product Skills…Your first Growth hire

"Growth” teams & PMs

…What is Product-led Sales…Shifting your perspective from users to teams… Product-qualified Accounts &amp; Leads…Expansion Revenue

Combining PLG with Sales: PLS

We use the majority of the time to discuss as a group your specific challenges and questions you had from the async learning materials and to address Q&amp;A questions.

6 x one - hour long sessions with Leah

Before every session you get ~30min videos to ease you into the topics including some interactive exercises for you to prepare beforehand.

Asynchronous learning material

Miss a session? Not a problem, you will have access immediately afterwards.

All sessions are recorded

Senior level product, growth &amp; marketing leaders in B2B who want to understand and leverage PLG as a distribution model to optimize growth.

Ideal for companies in B2B with some revenue traction post product-market fit.

GTM leaders wanting to understand why PLG is disrupting many classical distribution models.

Who is this course for

What you’ll learn

<ul><li>Review the core principles of product-led growth (PLG), beyond just trials and freemium</li></ul> <ul><li>Look under the hood of leading industry examples</li></ul>

Understand PLG and its impact on B2B growth initiatives

<ul><li>How to objectively measure customer success and operationalize it with meaningful, outcome-driven team goals</li></ul> <ul><li>How to establish growth in your organization that sticks with growth teams and culture</li></ul>

Increase the efficiency of your distribution model

<ul><li>How to move from activating individuals to teams and subsequently to accounts.</li></ul>

Individual vs. Team vs. Account Activation

<ul><li>How to find, define, and identify meaningful ICPs (Ideal customer profile)</li></ul> <ul><li>Driving them through our acquisition funnel and retaining them for long-term revenue</li></ul>

Find ICPs in Product-led Growth

<ul><li>How to combine the best from PLG and Sales-led Growth: Product-led Sales</li></ul> <ul><li>How to apply the principles of customer success signals, and their ICPs to qualify deals and drive efficiency in mid-market / enterprise segments.</li></ul>

Combine sales and Product-led Growth: Product-led Sales

<ul><li>Create a long-lasting Strategy to acquire customers while they are small and grow into bigger clients with time.</li></ul> <ul><li>Creating the conditions to have a product that grows with your customers</li></ul>

Drive expansion revenue through PLG

<ul><li>Community sessions to network and discuss with other cohort participants</li></ul> <ul><li>Access a dedicated Discord community after the course</li></ul>

Interactive community & special material

Understand Product-led Growth to formulate a strategy that includes PLG without wrecking your existing business.

Leah Tharin

Product & Growth executive / PLG Advisor

PLG Cohort 7

PLG Cohort 8

Want the exact playbook I used to scale brand at Brex and Notion? I'm opening up my complete toolkit—the frameworks, templates, and strategies that helped transform startups into category leaders worth $10B+. Where you are now:You're a brand or marketing leader juggling daily execution with long-term vision. Fighting for alignment while delivering results. Balancing need-it-now demands with strategic thinking. I've been there. As Brex's first marketing hire and former head of global brand at Notion, I learned what it takes to build lasting brands in hypergrowth environments. Now I'm sharing everything—including what worked, what failed, and what I wish I'd known sooner. This course is for you if:• You've outgrown generic brand playbooks and need battle-tested strategies• You're ready to evolve from tactical executor to strategic leader• You want to speak the language of growth and revenue• You aim to build brand equity, not just ship campaigns• You need frameworks for earning cross-functional buy-in What you'll get from me:The real inside tools• Step-by-step methods for winning executive buy-in• Campaign briefs (and templates) that drove 30% awareness growth• Resource planning docs that scaled through company growth from 50 to 1,500 employees• Implementable Notion templates for everything from brand refreshes to customer surveys• Frameworks for messaging, resourcing, and strategic planning Direct support &amp; community• 1:1 consulting sessions to tackle your specific challenges• Real-time feedback on your projects and strategies• Always-on Q&amp;A access to me and your cohort• A network of peers facing similar challenges• Lifetime access to course materials and updates The ExperienceJoin an intimate cohort of ambitious brand leaders for 4 weeks of intensive, practical learning. You'll walk away with:• Concrete tools to implement immediately• Strategic frameworks to guide long-term decisions• Confidence to lead brand at the executive level• A toolkit that scales as your company grows Limited to 15 seats per cohort to ensure personalized attention.

Inside Look: Building Multi-Billion Dollar Tech Brands 



Working in a dynamic landscape means constantly shifting priorities and an ever-evolving landscape. We'll orient ourselves in what success means—and how that changes over time. Overview:<ul><li>Define success: The pillars to scaling a brand</li><li>Identify critical brand opportunities in context of the business</li><li>Accessing Notion templates &amp; resources</li></ul>

Pre-work: Brand Strategy Milestones

Are you working on the right thing? Figure out your brand's biggest opportunities by diving into a case study from Brex on building foundational alignment. Notion Templates:<ul><li>Taste-Making Exercises</li><li>Brand Discussion Guide</li><li>Corporate Strategy Summary</li></ul>

Week 1: How to Find the Gaps

You've heard that consistency builds a brand. What builds consistency?Alignment—everyone needs to be on the same page about what you're building. We'll continue the Brex case study that led to their successful rebrand. Notion Templates:<ul><li>2x2 Grid: Brand Differentiation</li><li>Competitive Positioning Framework</li><li>Brand Foundations Framework</li></ul>

Week 1: How to Close the Gaps

The best ideas in the world don't matter without buy-in—or $ investment—from your leadership, your peers, or your team. This is how you get that buy-in. Notion Templates:<ul><li>Persuasive Brand Budget Requests</li><li>Brand Budgeting Spreadsheet</li><li>Pre-Mortem Brainstorms</li></ul>

Week 2: How to Get Buy-In

Build a multi-year marketing or campaign calendar that keeps all GTM teams aligned and informed on what you're putting out in front of customers. ﻿Notion Template:<ul><li>The Ultimate Marketing/Campaign/Launch Calendar</li></ul>

🔓 Bonus: Your marketing calendar needs a makeover

Craft brand-led, industry-leading campaigns that leverage the entire GTM function, with Notion as a case study. Notion Templates:<ul><li>Crafting an Effective Campaign Playbook</li><li>Define Internal SLAs for Cross-Functional Alignment</li><li>The Ultimate Campaign Brief (International, Multi-Channel)</li><li>Strategy Framework Library</li></ul>

Week 2: How to Build Winning Brand Campaigns

There are a thousand ways to measure a brand's impact, not all of them useful or practical. This module emphasizes strategies to glean actionable insights about your brand that help quantify its impact and inform future investments. Notion Templates:<ul><li>Social Proof Survey</li><li>Brand Health Survey Questions</li><li>Sourcing Brand Insights</li></ul>

Week 2: Actionable Brand Insights

Rebrands are often the culmination of all the skills referenced in the syllabus above. This bonus content, based on Brex's successful rebrand, covers tactical resources for maintaining alignment all the way through launch and beyond. Notion Templates:<ul><li>The Ultimate Rebrand Tracker</li><li>Internal FAQ for Rebranding</li></ul>

🔓 Bonus: Leading a Rebrand

My personal tech vendor roster is my rolodex—essentially my shortlist—of agencies and partners I've worked with or been in talks with over the course of my career, giving you an easy resource when you're ready to get work out the door! Even better, it's a Notion template. 😊

🔓 Bonus: Access my tech vendor roster

Each week features 1-2 modules plus bonus content. Each module includes a mix of projects, lessons (long-form and video), and templates that you can use right away.

Self-paced lessons

Each module pairs with a handful of practical projects, worksheets, or reflections to help bring the course content to life. Every project is an opportunity for discussion and feedback from Kira.

Practical projects

Join Kira for two live, 1:1 consulting sessions to ask questions about the course or get real-time feedback or advice. Kira will also respond to questions and provide feedback throughout the cohort async. 🕘 Tuesdays, between 9am-12pm PT / 12pm-3pm ET

Office hours

VPs &amp; heads of Marketing at fast-changing companies&nbsp;developing your brand strategy and growth levers alongside founders &amp; C-suite

Mid- to senior-level brand marketing ICs ready to find your path to leadership and up-level your org's brand strategy

Experienced marketing leaders&nbsp;seeking to refresh and deepen your tactical approach to modern brand strategy and campaigns

Senior leaders who are experts in other areas&nbsp;of marketing, like PMM or Growth, and want to get stronger in Brand

⚠️ This is not a Branding 101 course—we're going beyond the fundamentals and deep into daily operating rhythms

For Early & Growth-Stage Brand Leaders

What you’ll get out of this course

You'll get total access to my paid and unreleased <a href="https://www.notion.so/@kira" rel="noopener noreferrer" target="_blank">Notion templates</a>, created from the frameworks I've developed over 10+ years of operating brands at growing tech companies like Gusto, Brex, and Notion—plus my in-depth voiceover and context on how to get the most from them.

25+ real-life templates for scaling a world-class brand (that you can use right away) 

Apply actionable brand measurement strategies at all stages of growth, yielding insights you can actually use. Proactively navigate conversations with senior leadership so they invest in brand, keeping the brand front and center throughout organizational shifts &amp; planning cycles.

Create the buy-in & alignment you need to build a loved brand that drives growth

25+ proven templates from hypergrowth startups: Get executive buy-in, build winning campaigns, and prove brand value from day one