Simple AI Labs

Stop Shipping AI Agents on Vibes

Uber

Adobe Marketo Engage

Simplified

Arizona State University

KD Deshpande

AI Founder, Building the Future of AI Agents & Agentic Workflows.

Why "it passed our tests" isn't enough; the PM/Legal/Ops trust gap (origin story).

The vibes problem

What an eval actually is: input → expected behavior → judge. The 4 kinds that matter: correctness, quality, compliance

The mental model

Catch agent failures before your users (or your customers) do.  Leave with a reusable framework.

Hands-on

Build a real AI agent, find where it breaks, and improve it with evals you can trust, working the full loop hands-on.

ML Engineer with 25 years of experience

Airbnb

GitHub

DataRobot

AlixPartners

Hamel Husain

ML Engineer with 20 years of experience.

ML Systems & Applied AI Evals Researcher

Google

UC Berkeley

Stanford University

Shreya Shankar

ML Systems Researcher Making AI Evaluation Work in Practice

Cohort 5

Cohort 6

🏆 The most field-tested evals course available. We've refined this exact material over a year of cohorts with 4,500+ engineers and PMs from teams like OpenAI, Google, Meta, Amazon, and Microsoft, folding their feedback in every time. <a target="_blank" rel="noopener noreferrer nofollow" href="https://testimonial.to/ai-evals-course/all">Read what students say →</a>More than a course: an ongoing system, tools, and community for shipping better AI.<ul><li>🎮 A private Discord community for ongoing support, even after the course.</li><li>🤖 6 months of access to our AI Evals assistant.</li><li>♾️ Lifetime access to all recordings, materials, and future cohorts.</li><li>💬 10+ hours of live office hours to get your questions answered.</li></ul>Do you catch yourself asking any of these while building AI applications?<ol><li>How do I test outputs that need subjective judgment?</li><li>If I change the prompt, how do I know I am not breaking something else?</li><li>Where should I focus my efforts? Do I need to test everything?</li><li>What if I have no data or customers yet? Where do I start?</li><li>What should I measure, and what tools should I use?</li><li>Can I automate evaluation, and how do I trust it?</li></ol>If so, this is for you. All sessions are live and recorded.

 Build, evaluate, and improve AI agents that work in production.

Build an agent you can measure

Find what's broken with error analysis

Build evaluators you trust

Catch regressions with CI/CD

Red-team for safety before users do

Improve accuracy and cut cost with evidence

Engineers and PMs who ship prompt changes and hope nothing breaks. (You'll learn to measure impact before and after every change.)

Teams still spot-checking AI outputs by hand instead of measuring systematically. (You'll learn how build automated evals you can trust.)

Leaders who don't know where their AI is failing or where to invest resources. You'll learn how to systematically find &amp; prioritize issues.

Build AI Agent Skills that Actually Work - 3 Hours Hands-on Workshop

Product Leader and Super IC at Google

Disney+ Hotstar

Zynga

Anshumani Ruddra

Product Super IC at Google with 22+ years crafting experiences and blockbuster products

Cohort 03

Cohort 04

Most product builders use AI agents the same way they used search - typing queries and hoping for the best. That's leaving most of the value on the table.The teams pulling ahead aren't using better prompts. They're teaching their agents. Agent Skills are reusable, testable instruction sets that tell AI agents exactly how to handle specific workflows: your deployment process, your code review standards, your team's edge cases. Write one skill well, and every agent on your team benefits from it, every time, automatically.The problem: nobody is teaching product builders how to create them. Most documentation is written for a narrow technical audience, and the tooling is fragmented across Claude Code, Codex, and Gemini.In this hands-on workshop, you'll go from zero to a working skill installed in your own agent: one that reflects your real workflows, not generic templates. You'll also learn how to evaluate and improve skills systematically, so they get better over time instead of drifting.If you build or are hoping to build products with AI agents: this is the highest-leverage skill you can develop right now.

🤖 Learn to build procedural Agent Skills, transforming your professional mindset from solo prompter to manager of elite AI agents

Shift to an AI Management Mindset

Build Custom, Reusable Agent Skills

Leverage the Open-Source Ecosystem

Evaluate and Iterate for 10x Output

Product Managers: who use Claude Code or Codex daily but rely on ad-hoc prompts - and want repeatable, team-shareable AI workflows.

Entrepreneurs &amp; Founders: building AI-native products who need their agents to encode business logic, not just generate code on demand.

Tech Executives &amp; Product Leaders: who want their teams to stop reinventing prompts and start shipping a shared skills library.

Eliminate uncertainty in shipping AI features

GenAI Product Builder @ Google, ex-Meta

Marily Nika, Ph.D AI/ML

Senior Data, ML and GenAI Scientist

George Zoto

Senior NLP/AI Scientist

Cohort 4

“Does it work?”... “Is it good enough?”... “Can we ship it?”...How do you answer these questions for AI products? You’re responsible for “running evals” but what does that mean?How do you choose the right metrics, interpret fuzzy results, and make a confident decision?This course gives you a framework to do just that.<ul><li>Map user value to evaluation (eval) objectives so your metrics aren’t abstract. Define success then translate it into measurable criteria.</li><li>Choose metrics you can actually maintain: capability, safety, UX friction, latency, cost and “does this reduce support tickets or increase activation.”</li><li>Set ship/no-ship thresholds you can defend to leadership.</li><li>Build lightweight workflows that work in real teams: human review where it matters, automation where it lasts, documentation that drives decisions.</li><li>Consider domain constraints (e.g., healthcare safety) and know what to avoid: silent failures, misleading proxy metrics and tests that don’t reflect production.</li><li>Tie everything to ROI: impact vs unit cost, eval coverage vs reliability, and the minimum viable monitoring you need post-launch.</li></ul>Experience AI evals through a case-based approach with a real AI product that we evaluate together.

Acquire and develop a critical skill for product managers who are leading and contributing to AI products.

Make confident ship or hold decisions for AI features

Translate user value into clear evaluation goals

Choose the right metrics for capability, safety, UX, and cost

Set defensible thresholds leadership will trust

Build lightweight evaluation workflows teams can maintain

Navigate domain constraints and avoid common failures

PMs leading AI features, growth, or platform initiatives

PMs who partner with ML teams and want to set evaluation standards

PMs who need to make clear “ship or hold” calls without doing the engineering

Build and deploy a real-time AI agent with production-ready patterns you can reuse in your own products.

Build & Deploy AI Agents

Amazon

Prime Video

Cruise Norwegian

Zurich

Emilio Taylor

Author of Vibe Engineering | Founder, Visao + Helix | AI Systems for RevOps

First public cohort focused on building and shipping real-time AI agents using LiveKit and Supabase.

Fall 2026 Cohort

<blockquote>Most AI tutorials stop at prompts and demos.</blockquote><blockquote>But real AI systems don’t fail at prompts — they fail in execution.</blockquote><blockquote></blockquote><blockquote>Without workflow state, tool boundaries, retries, and observability, AI features break quickly in production.</blockquote><blockquote></blockquote><blockquote>In this course, you’ll build and deploy a real-time AI agent using LiveKit and Supabase — not just a demo, but a working system.</blockquote><blockquote></blockquote><blockquote>You’ll learn how to design AI as workflows, connect tools safely, stream responses in real-time, and handle failure cases like a production engineer.</blockquote><blockquote></blockquote><blockquote>By the end, you’ll have a deployed agent and a repeatable blueprint you can use in your own product.</blockquote>

Most AI tutorials stop at prompts. This one ships systems.

Build your first AI agent (end-to-end)

Design reliable AI workflows (not just prompts)

Add real-time capabilities with LiveKit

Ship production-ready AI systems

You’ll extend a real codebase, run locally, and ship features quickly without getting blocked on frontend basics.

Comfortable building React apps locally

Your agent will read and write real data. You should be comfortable working with APIs and basic database patterns.

Basic APIs + database knowledge (CRUD)

This is a code-first course. You’ll read logs, handle failures, and debug behavior to make your AI agent reliable.

Willing to write JS/TS and debug issues

<blockquote>Full-stack engineers (React/JS) who want to build real AI agents—not just prompt demos—and ship systems that actually work in production.</blockquote>

Indie builders who want to quickly build and launch an AI-powered product or agent with a real backend and scalable architecture.

<blockquote>Product engineers and engineering leads looking to establish reliable patterns for building AI agents inside real applications.</blockquote>

Agentic AI

Coding with AI

AI Workflows

Claude Code

OpenClaw

Vibe Coding

AI Evals

AI Transformation

RAG & Search

AI for PMs

AI for Engineers

AI for Designers

AI for Marketers

AI for Founders

Product Sense

Product Discovery

User Research

Prototyping

Growth

Analytics

Tech Foundations

Strategy

Influence

Leadership

Career Growth

Machine Learning

LLM Ops

Context Eng

Security

System Design

Figma

Design Systems

Visual Design

Design Strategy

Automation

Content Marketing

Demand Gen

Go-to-Market

Product Marketing

Positioning

Social Media

Brand

B2B Marketing

SEO & AEO

AI for Leaders

AI Governance

Communication

Management

People Operations

Exec Presence

Storytelling

Goal-setting

Personal Brand

Fundraising

Everyone

Operators

Data Scientists

Business Analysts

User Researchers

Customer Success

Project Managers

HR Professionals

Sales People

Lawyers

Finance

Investors

Real Estate

Educators

Creators

Product managers who drive strategy, development, and launch of products.

Engineers and engineering leaders who build, deploy, and scale software systems.

Applying AI tools and techniques to professional workflows — assign alongside more specific AI topics.

Building autonomous AI agent systems that take actions, use tools, and orchestrate multi-step workflows.

Measuring and evaluating AI model performance, quality, and reliability for production use cases.

Operating and deploying large language models in production — fine-tuning, inference, monitoring, and cost optimization.