Learn Agentic AI: Setting agents metrics and evaluations

Lightning Lessons

Practical skills & tools to accelerate your career

Learn Agentic AI: Setting agents metrics and evaluations

Hosted by Mahesh Yadav

749 students

What you'll learn

Challenges in measuring & evaluating AI

Traditional metrics fall short for AI, making evaluation complex and often misleading.

Metrics specific to AI performance

We cover how to set metrics for AI agents (goal specific) and create evaluation that measure AI agent performance.

Automating AI evaluation using LLMs as judges

Leverage AI to evaluate AI—we will cover how you can use LLM as judges, automate evals with BLEU and ROUGE like metrics.

Why this topic matters

Everyone is building AI agents, giving you many options for handling tasks like content creation, lead generation, or research. But how do you evaluate these agents to select one that truly delivers value? Setting the right metrics will help you make steady progress toward your goals and keep your team focused on results, not hype.

You'll learn from

Mahesh Yadav

Gen AI product lead at Google, Former at Meta AI, AWS AI l 10k+ AI PM Students

Mahesh has 20 years of experience in building products at Meta, Microsoft and AWS AI teams. Mahesh has worked in all layers of the AI stack from AI chips to LLM and has a deep understanding of how using AI agents companies ship value to customers. His work on AI has been featured in the Nvidia GTC conference, Microsoft Build, and Meta blogs.

His mentorship has helped various students in building Real time products & Career in Agentic AI PM space.

Previously at