Validate Claude Code Analytics Output

Shane Butler: Principal Data Scientist at Ontra

Sravya Madipalli: Senior DS Leader (Ex-Microsoft)

Vibe Code Annotation UIs for AI Analytics Evals

Shane Butler: Principal Data Scientist, AI Evaluations at Ontra

Ship a Production Cursor Agent System in 30 Minutes

Carmelo Iaria: Founder at Synaptic AI Consulting | O'Reilly Instructor | Pearson SME | ex-Cisco

Debug Cursor Agent Failures Before Production

Carmelo Iaria: Founder at Synaptic AI Consulting | O'Reilly Instructor | Pearson SME | ex-Cisco

Raise Your Technical Bar as an AI-Native PM

Jason P. Yoong: Co-Founder & COO at Level Up (Prev: Amazon, Dentsu, IPG, Startup - $8M seed)

Gayathri Keerthana (GK): AI and Product Consultant | ex-Amazon Product Leader | ESADE MBA

No Vibes, Just Evals: Proven Frameworks for AI-Native PMs

Jason P. Yoong: Co-Founder & COO at Level Up (Prev: Amazon, Dentsu, IPG, Startup - $8M seed)

Shankha Dey: Sr Director PM @ Salesforce | ex-Microsoft Bing & Copilot | Columbia CS

Modern Information Retrieval Evaluation In The RAG Era

Nandan Thakur: RAG researcher @ UWaterloo. Creator of BEIR and MIRACL benchmarks

Hamel Husain: ML Engineer with 20 years of experience

Debug the weird stuff your AI does (in less than 1 hour)

Marily Nika: AI Product @ Google, Founder of AI Product Academy

Hamel Husain: ML Engineer with 20 years of experience.

Mastering Evals as an AI Product Manager

Aman Khan | AI PM Operator and Expert: AI PM (Player + Coach) with ~10 years of hands-on product experience

Automating Evals With Claude Code + Phoenix

Mikyo King: Head of Open Source. Building the future of AI Observability at Arize AI

Hamel Husain: ML Engineer with 20 years of experience

AI Evals for Product Managers

Anshumani Ruddra: Product Leader and Super IC at Google

How to Setup Evals For Agents

Harrison Chase: Co-Founder and CEO at LangChain

Hamel Husain: ML Engineer with 20 years of experience

Error Analysis: The AI Engineer’s Best ROI

Hamel Husain: ML Engineer with 20 years of experience.

Shreya Shankar: ML Systems Researcher Making AI Evaluation Work in Practice

Evaluating AI Agents

Amir Feizpour: Founder @ Aggregate Intellect

Samuel Dion-Girardeau: CTO at Tilt

Evaluating Agentic AI Applications Beyond Vibe Checks

Aishwarya Naresh Reganti: Tech Lead @ AWS | AI Lecturer & Advisor

Kiriti Badam: Member of Technical Staff @ OpenAI | Ex-Google

Understanding Embedding Performance through Generative Evals

Jason Liu: Consultant at the intersection of Information Retrieval and AI

Kelly Hong: Researcher at Chroma

How OpenAI Customers Use Evals To Build Better AI Products

Jim Blomo: Engineering Leader at OpenAI, working on Evals

Hamel Husain: ML Engineer with 20 years of experience

Evaluating LLMs for Your Applications

Mahesh Yadav: GenAI Product Lead at Google, previously at Meta, Amazon, and Microsoft

How Evals Made GitHub Copilot Happen

John Berryman: ML Researcher, Software Engineer, and Author; Worked on GitHub Copilot

Shawn Simister: Member of Technical Staff, Factory

Setting Eval for AI Agents & Scaling with Auto-Evaluation

Mahesh Yadav: Ex- GenAI Product Lead at MAANG Firms l AI PM Coach l 10k+ Alumni

Learn Agentic AI: Setting agents metrics and evaluations

Mahesh Yadav: Gen AI product lead at Google, Former at Meta AI, AWS AI l 10k+ AI PM Students

Optimize Structured Data Retrieval With Evals

Daniel Svonava: CEO at Superlinked. Previously ML TL at YouTube.

Hamel Husain: ML Engineer with 20 years of experience

Online Evals and Production Monitoring

Jason Liu: Consultant at the intersection of Information Retrieval and AI

Ben Hylak: Co-Founder, Raindrop

AI Systems Under Pressure: Red-Team Before You Ship

Krystal Jackson: AI Standards Development Researcher

Evaluate AI agents with Confidence

Mahesh Yadav: Gen AI product lead at Google, Former at Meta AI, AWS AI, 10k+ AI PM Students

Design Evals Users Will Trust

Aishwarya Naresh Reganti: AI Founder & Advisor to F500 Leaders

Improve reliability of your AI applications

Shreya Rajpal: CEO and Cofounder, Guardrails AI

Optimize Your Dev Setup For Evals w/ Cursor Rules & MCP

Isaac Flath: AI Engineer & Fullstack Developer

Hamel Husain: ML Engineer with 20 years of experience

Build Your Own Eval Tools With Notebooks!

Vincent D. Warmerdam: Engineer @ marimo

Hamel Husain: ML Engineer with 20 years of experience

Evaluation Driven Development for Agentic AI Systems

Aurimas Griciūnas: LinkedIn Top Voice in AI • Founder & CEO @ SwirlAI • Former CPO @ Neptune.ai

Build Your AI Evals & Analytics Playbook

Stella Liu: Head of AI Evals at ASU

Amy Chen: Co-Founder at AI Evals & Analytics

How You Catch Production Hallucinations in Real Time

Jason Liu: Consultant at the intersection of Information Retrieval and AI

Julia Neagu: Co-founder & CEO, Quotient AI

Production Grade AI Evals by Braintrust.dev

Mengying Li: Mengying Li, Head of Data at Braintrust.dev

Evaluation-Driven Development for AI Products

Roger Jin: ex-Product Lead for Gemini@Google DeepMind, 2x Founder, AI Strategy Coach

Scaling Judge-Time Compute for Robust Auto LLM Evaluation

Jason Liu: Consultant at the intersection of Information Retrieval and AI

Leonard Tang: Co-Founder & CEO @ Haize Labs

How to Find Unexpected AI Outcomes

Kerry Bodine: Co-author of OUTSIDE IN, international keynote speaker, coach and consultant

Dan Saffer: Assistant Professor of The Practice at Carnegie Mellon University and Author

Practical Evaluation Strategies for AI Agents

Hamza Farooq: Founder | Ex-Google | Adjunct UCLA & UMN, SCU | Venture Partner

Gabriela de Queiroz: Ex-Microsoft & IBM AI leader | AI Advisor for Startups

The AI Evals Career Fast Lane - 30-Minute On-Ramp

Srini Annamaraju: Ex-Capgemini/HPE/TCS, 25+ yrs in tech, Chair and co-founder working on AI trust.

Shen Pandi: 20+ Years in Data & Tech Consulting. Ex-Deloitte, McKinsey, EY. Founder & CEO.

Strategies for building self-improving document processing

Jason Liu: Consultant at the intersection of Information Retrieval and AI

Eli Badgio: Co-founder and CTO at Extend

Master Evaluation Techniques for LLM Apps

Haroon Choudery: CEO, Autoblocks AI

AI Evals for Product Managers

Anshumani Ruddra: Product Leader and Super IC at Google

AI Evals vs. Product Metrics: A 30min Crash Course

Stella Liu: Head of Applied Scientist, AI Evals

Design an AI Quality Dashboard for PMs

Aki Wijesundara, PhD: AI Founder | Educator | Google AI Accelerator Alum

Manu Jayawardana: AI Advisor | Co-Founder & CEO at Krybe | Co-Founder of Snapdrum

Evaluate the quality and accuracy of AI insights

Ann Conway: Founder & CEO of Amplify, helping brands use AI to transform their work.

How to Drive AI Evals Adoption

Dr Sebastian Fox: CEO & Co-founder at Composo.ai

Validate Business Impact of AI Features

Shane Butler: Principal Data Scientist, AI Evaluations at Ontra

Understand SHAP (SHapley Additive exPlanations)

Patrick Hall: Principal Scientist, HallResearch.ai & Assistant Professor, GWSB

Reliable RAG Agents: Intent-Driven Failure Detection

Jason Liu: Consultant at the intersection of Information Retrieval and AI

Ben Hylak: Co-Founder, Raindrop

Create MCP Tool Evals Before You Ship

Emmanuel Paraskakis: Founder, Level 250 | 3x VP, API Product

Don't Tweak Prompts. Engineer Agents.

Hugo Bowne-Anderson: Podcaster, Educator, DS & ML expert

Skylar Payne: AI made easy. AI executive for startups. Ex-Google. Ex-LinkedIn.

Scale Evals Without the Chaos

Aishwarya Naresh Reganti: AI Founder & Advisor to F500 Leaders

Evals for Voice AI: Learnings from Google Evals Team

Ravin Kumar: Sr. Researcher at Google Deepmind

Mastering LLM Application Testing

Hugo Bowne-Anderson: Podcaster, Educator, DS & ML expert

Stefan Krawczyk: 13+years in MLOps: Ex-Stitch Fix, Ex-Nextdoor, Ex-LinkedIn

Synthetic RAG evaluation

Alexey Grigorev: Principal Data Scientist | Book Author | Instructor to 100k+ Students World-Wide

Doug Turnbull: Principal ML Engineer in Search

Calibrate LLM-as-a-judge for Real-world Impact

Eddie Landesberg: Founder of CIMO Labs

Evals in Action With Arize

Laurie Voss: Head of DevRel at Arize, co-founder, npm Inc

Redesign Your Product Metrics for AI Evals

Shane Butler: Principal Data Scientist, AI Evaluations at Ontra

Bring AI Evals to Product Leadership

Stella Liu: Head of Applied Scientist, AI Evals

Amy Chen: Co-Founder at AI Evals & Analytics

🛠 Synthetic Data Flywheels: Build Reliable LLM Apps Faster

Hugo Bowne-Anderson: Podcaster, Educator, DS & ML expert

Stefan Krawczyk: 13+years in MLOps: Ex-Stitch Fix, Ex-Nextdoor, Ex-LinkedIn

The Hidden Signal in Production AI Logs

Jason Liu: Consultant at the intersection of Information Retrieval and AI

Scott Clark: Co-Founder & CEO, Distributional

Master AI Hallucinations: Turn Glitches into Breakthroughs

Abi Awomosu: Founder & CEO of Data, Art & Soul (Ex-Uber, Apple, Meta, Microsoft)

How to test and improve your AI agents

Jacob Bank: Founder, Relay.app. 65k+ Followers on LinkedIn.

Pragmatic AI Evals: Beyond Vendor Promises

Stella Liu: Head of Applied Scientist, AI Evals

Amy Chen: Co-Founder at AI Evals & Analytics

De-Risking LLM Model Switches w Evals & Prompt Optimization

Amir Feizpour: Founder @ Aggregate Intellect

Hugo Mailhot: CoreNLP Lead @ Coveo

Part 3: Building Robust Evaluations for AI Agents

Hamza Farooq: Founder & Adjunct Professor | 15+ years | Google | Stanford | UCLA

Gabriela de Queiroz: Ex-Microsoft & IBM AI leader | AI Advisor for Startups

Evaluate the quality and accuracy of AI insights

Ann Conway: Founder & CEO of Amplify, helping brands use AI to transform their work.

Build a Career Kit for the AI Evals Hiring Surge (45 mins)

Srini Annamaraju: Ex-Capgemini/HPE/TCS, 25+ yrs in tech, Chair and co-founder working on AI trust.

Shen Pandi: 20+ Years in Data & Tech Consulting. Ex-Deloitte, McKinsey, EY. Founder & CEO.

Collaborative AI Evals with Human Feedback

Rogério Chaves: CTO & Cofounder at Langwatch

Stay Ahead in AI: Evaluate Any New LLM in 15 Minutes

Sherveen Mashayekhi: Founder & CEO @ Free Agency, AI Product Leader & Investor

Evaluating AI Agents before Users Break Them

Aki Wijesundara, PhD: AI Founder | Educator | Google AI Accelerator Alum

Marc Klingen: Co-Founder & CEO of Langfuse

Run Eval Loops and Guardrails for Cursor Agents

Carmelo Iaria: Founder & CEO at Synaptic AI Consulting | Perplexity Business Fellow

Setting up your first AI eval with a LLM-as-judge

Madalina Turlea: Co-founder @Lovelaice, 10+ years in Product

Catalina Turlea: Founder @Lovelaice

Go Beyond AI Evals: Diagnose and Decide

Rajiv Shah: AI Engineer, 10+ years, 100k followers

From AI user to AI feature designer: a live exercise

Madalina Turlea: Co-founder @Lovelaice, 10+ years in Product

Catalina Turlea: Founder @Lovelaice

How to test AI when you don't have any data yet

Madalina Turlea: Co-founder @Lovelaice, 10+ years in Product

Catalina Turlea: Founder @Lovelaice