Build AI Evals for Your Own AI Product From Scratch, Hands-on

Madalina Turlea

Co-founder @ Lovelaice, Product Leader

Catalina Turlea

2x founder | CTO | 14 years in Tech

Run the Full Evals Loop on your own AI product or agent, with weekly expert help

Anyone can build an AI feature or agent in a few days now. The hard part is knowing if it works for your users before they tell you it doesn't. The best AI products are built when PMs shape the AI and its evals directly, not when it's left to engineering alone.

This course gives you a way to answer it with numbers instead of gut feel, whether your feature is live or still on the roadmap. You'll get answers to the questions product teams ask us most:

- How do you test AI before production, when you don't have real data yet?

- How do you define "good" when it's different for each of your customers?

- How do you find where an agent that calls tools over many steps breaks?

- Which model should you ship once you weigh quality, cost and latency?

- How do you catch regressions every time someone changes the prompt?

- How do you choose an LLM judge, and know when you can trust it?

You bring your own AI product, feature or agent. Over four weeks, you run the full loop on it: build the dataset, compare models, find the failures, write the evals and ship the fix. We review your work live, every week.

Evals are a new habit, and habits come from reps on the product you're responsible for.

What you’ll learn

Measure and improve AI quality on your own product, and become the person your team trusts to decide when AI is ready to ship.

  • Turn your product decisions into clear instructions, the way you'd write a PRD

  • Decide how the AI handles edge cases (empty input, off-topic requests, missing data), so it doesn't decide for you

  • You leave with: a system prompt reviewed by us before you test it at scale

  • Turn the ways your users differ (new vs. experienced, language, industry) into typical, edge and "try to break it" test cases

  • Handle data that changes every day or at every run, and build it into a golden dataset you can rerun anytime

  • You leave with: your first golden dataset, built with our help

  • Run your feature on multiple models at once

  • Weigh quality, cost and latency together to build the business case for your choice

  • You leave with: a model comparison you can show your leadership

  • Review outputs and write specific notes, including on tool calls: right tool, called correctly, data read right

  • Group your notes into failure patterns and rank them by how much they hurt your users

  • You leave with: a ranked list of your product's real failures

  • Write a rubric from your real failures, so "good" reflects your customers and not a generic template

  • Start with free code checks, add LLM judges only where judgment is needed, and test each judge against your team's own labels

  • You leave with: your rubric and first set of evals, reviewed with us live

  • Link each failure to a fix, then re-run your evals to see what improved and what broke

  • Use the same loop to test new feature ideas before you build them

  • You leave with: before-and-after numbers and a loop you can repeat on your next AI feature

Learn directly from Madalina & Catalina

Madalina Turlea

Madalina Turlea

Product leader with 10+ years experience in FinTech | Founder &CPO Lovelaice

Catalina Turlea

Catalina Turlea

Founder & CEO @Lovelaice | Co-founder & ex-CTO @ nilo | 14 years in tech

See all products from Madalina

Who this course is for

  • Product managers who own an AI feature, live or planned, and want to define its quality themselves instead of waiting for engineering

  • Engineers building AI features who want a shared eval process with their PM, not a prompt that “seems fine”

  • Data scientists who want to help product teams choose the right eval metrics and test for real impact.

What's included

Live sessions

Learn directly from Madalina Turlea & Catalina Turlea in a real-time, interactive format.

Lifetime access

Lifetime access to session recordings and course materials

Lovelaice platform access

Lovelaice platform access to run your experiments and evals, with models included and no API keys needed

Bring your feature.

Direct feedback on your AI product from both instructors, every week

Community

A community of peers building AI products

Maven Guarantee

Your purchase is backed by the Maven Guarantee.

Course syllabus

4 live sessions • 5 lessons • 2 projects

Live sessions

1 hr / week

Once a week, we meet to build together your AI feature

Projects

1 hr / week

Dive deeper and run on your own after the session

Async content

1 hr / week

Bonus content, extra articles to enhance your learnings

Week 1

Jan 8—Jan 10

    Start here: Welcome to the course + Week 1: Define and build your feature

    3 items

    Jan

    8

    Session 1

    Fri 1/812:00 PM—1:00 PM (EST)

Week 2

Jan 11—Jan 17

    Week 2: Run it, read it, find the failures

    1 item

    Jan

    15

    Session 2

    Fri 1/1512:00 PM—1:00 PM (EST)

Free resources

Testimonials

  • I love the amazing sessions with Cata and Mada from Lovelaice. Me and my company are building a framework to develop prototypes and web applications with the help of AI. Being a non-technical person, I try to catch up with the developers of my team and that's where the workshops come into the picture. They help me a lot to understand a little bit more from the AI world week by week. Thank you girls so much!

    Testimonial author image

    Reka

    Start-up Founder at Fluxion
  • When I enrolled in this series, I didn’t expect to gain such a clear, structured approach to working with AI. It opened my eyes to the importance of systematic testing; no more half-baked prompt experiments or guesswork. Thanks to the lessons, I picked up practical techniques that helped me separate hype from real performance, and gave me a playbook I can apply confidently. I now approach AI testing methodically, and I feel far more equipped to judge what works and what doesn’t. This series truly shifted the way I work with AI, and I’d recommend it to anyone serious about getting real, reliable results.

    Testimonial author image

    Barbara

    Senior Fullstack Developer
  • Each session takes us deeper into the advantages of AI, from experiment evaluation to exploring how different models behave. I enjoy how your passion and clear explanations make prompt engineering, practical tips, and even industry insights so easy to understand. Learning from you both is a pleasure.

    Testimonial author image

    Maria

    Senior Developer
  • I always enjoy the sessions. I feel like even if I think I understand everything, every time feels like there is an aha moment.

    Furthermore, I also like how relatable the experiments are and how interesting and useful the results always are.

    Testimonial author image

    Pavlina

    Senior Frontend Developer

Frequently asked questions

Maven for Teams

Reimbursement

Get your company to pay

Everything L&D needs: email template, receipts, and certificate of completion.

Get reimbursed

Team discount

Learn with your teammates

Save 20%+ when 2 or more teammates enroll in the same cohort.

Save 20%+ with a team

Private cohort

Run a cohort for your org

A dedicated cohort with a custom schedule and curriculum, tailored to your team.

Book a private cohort

$2,190

USD

·
Jan 8—Jan 29
Enroll