Production-Ready Recommenders

Katerina Zanos

Principal ML Engineer @ Disney, ex-Meta

Learn the production lessons behind recommenders that actually work.

Build a production-ready Recommender System Blueprint you can take back to work to guide implementation, align your team, and make smarter roadmap calls.

Most recommender projects don't fail because of the algorithm. They fail because a recommender learns from its own decisions, and teams don't design for that:

  • The metric you pick reshapes the product. Optimize the wrong number and the system will find the cheapest way to move it.

  • Your data is a record of what you chose to show. Items you never showed can't teach your model anything.

  • A better model often changes nothing. The bottleneck is usually somewhere else.

  • A model can win the test and flip after launch. The world it was tested in isn't the world it creates.

These lessons usually take years of production mistakes to learn. I've made plenty of them over 10 years building recommenders at The New York Times, Meta (Feed & Reels), and Disney/ESPN. This course distills them into five focused weeks.

What you’ll learn

Learn to design and evaluate a production-ready media recommender: metrics, architecture, cold start, and drafting an effective roadmap.

  • Turn a vague goal into a metrics spec: one number to move, and the few that must not break.

  • Spot the traps that quietly hurt products: clickbait, sugar highs, and cannibalization.

  • Write success criteria that engineers, PMs, and leadership actually align on.

  • Understand why showing is sampling, and how exposure bias corrupts your data.

  • Design logging that can answer "why did this user see this?"

  • Plan traceability and fallbacks that keep the system debuggable and reliable.

  • Diagnose whether your bottleneck is retrieval, ranking, or logging.

  • Decide what to predict (the label) before deciding how to predict it.

  • Make a defensible model bet using an expected-lift scorecard.

  • Learn why LLMs strengthen the stack instead of replacing the ranker.

  • Use LLMs to make new content recommendable from day zero.

  • Add exploration so agents don't just recommend what's already trending.

  • Break big bets into milestones where each step earns the next.

  • Know when ML beats heuristics, and prove it with the smallest credible test.

  • Use backtests and holdouts to check that wins still hold after launch.

Learn directly from Katerina

Katerina Zanos

Katerina Zanos

Principal ML Engineer @ Disney, x-Meta, x-NYT

Meta
The New York Times
The Walt Disney Company
ESPN
Columbia University
See all products from Katerina Zanos

Who this course is for

  • Software Engineers moving into ML and recommendations who want the systems and measurement mindset, not just training code.

  • ML Engineers and Data Scientists who can build models but want sharper judgment on what to build next and how to defend it.

  • Product Managers who own a recommendation surface and want to set the right metrics and make roadmap calls with their ML team.

Prerequisites

  • Basic metric and product reasoning.

    You can interpret common product metrics like retention and engagement and discuss tradeoffs.

  • Ability to follow system-level thinking.

    You can follow an architecture diagram and understand services, latency, and reliability.

  • No deep coding experience required.

    Each assignment has a build step where you use AI coding tools to run a working solution. Engineers get stretch goals to go deeper.

What's included

Katerina Zanos

Live sessions

Learn directly from Katerina Zanos in a real-time, interactive format.

Lifetime access to 5 self-paced lessons with practical exercises

Absorb core concepts on a schedule that works for you. Walk into live sessions ready to dive deeper, apply lessons, share progress, and workshop challenges.

Lifetime membership to a community of ambitious builders

Stay accountable, share insights with like-minded community builders throughout the course and beyond in a community led by Katerina on Maven.

Weekly office hours (60 min).

Work through the assignment live with Katerina and your peers: debug, compare approaches, and apply the lesson to your own product.

Direct 1:1 async access to Katerina for Q&A

Throughout the 5 week course, message Katerina anytime with questions specific to your own problem sets and goals.

Certificate of completion

Share your new skills and signal that you've committed to being the best in your craft.

Maven Guarantee

Your purchase is backed by the Maven Guarantee.

Course syllabus

10 live sessions • 15 lessons • 5 projects

Week 1

Oct 21—Oct 25

    Oct

    21

    What You Reward Is What You Get

    Wed 10/213:30 PM—4:30 PM (UTC)

    Lessons: The Cheapest Way to Move a Number · Finishing a Metric

    4 items

    Oct

    23

    Build Lab: Your Metrics Spec

    Fri 10/234:00 PM—5:00 PM (UTC)
    Optional

Week 2

Oct 26—Nov 1

    Oct

    28

    You Can Only Learn From What You Showed

    Wed 10/283:30 PM—4:30 PM (UTC)

    Lessons: Showing Is Sampling · Why Did This User See This?

    4 items

    Oct

    30

    Build Lab: Your Engine Blueprint

    Fri 10/304:00 PM—5:00 PM (UTC)
    Optional

Free resource

Kickstart Your Recommender System cover image

Kickstart Your Recommender System

Understand the core building blocks of a recommender system

and how metrics connect them.

Use metrics to guide recommender system design

and shape architecture and engineering decisions

Build a metrics stack that reflects your product goal

and move from vague objectives to concrete levers you can pull.

Schedule

Live sessions

2 hrs / week

    • Wed, Oct 21

      3:30 PM—4:30 PM (UTC)

    • Fri, Oct 23

      4:00 PM—5:00 PM (UTC)

    • Wed, Oct 28

      3:30 PM—4:30 PM (UTC)

Assignments

1-2 hrs / week

Use a real use case from your work for the assignment or if you don’t have a use case ready, you can use the provided media case study instead.

Frequently asked questions

Maven for Teams

Reimbursement

Get your company to pay

Everything L&D needs: email template, receipts, and certificate of completion.

Get reimbursed

Team discount

Learn with your teammates

Save 20%+ when 2 or more teammates enroll in the same cohort.

Save 20%+ with a team

Private cohort

Run a cohort for your org

A dedicated cohort with a custom schedule and curriculum, tailored to your team.

Book a private cohort

$1,300

USD

·
Oct 21Nov 20
Enroll