Nikhil Pentapalli

Senior AI Engineer | GenAI, RAG, Agents | Adobe | Ex-Startups | 9 Years in AI

Build and Evaluate an AI Code Reviewer Agent

Adobe

O'Reilly Media

Cohere

Analytics Vidhya

Bold Penguin

Build a working AI agent that connects to GitHub, fetches PR diffs, and generates line-level review comments

Build an AI Code Reviewer Agent
Description

Design prompts that produce specific, actionable feedback  not the generic "this looks fine" a basic LLM call gives

Prompt Design

Understand how to evaluate your reviewer and know where it breaks before you put it in front of your team

Evaluate your agent

Build a real AI agent, find where it breaks, and improve it with evals you can trust, working the full loop hands-on.

ML Engineer with 25 years of experience

Airbnb

GitHub

DataRobot

AlixPartners

Hamel Husain

ML Engineer with 20+ years of experience.

ML Systems & Applied AI Evals Researcher

Google

UC Berkeley

Stanford University

Shreya Shankar

ML Systems Researcher Making AI Evaluation Work in Practice

Cohort 5

Cohort 6

Save 25% ($1,050) on September's cohort <a target="_blank" rel="noopener noreferrer nofollow" href="https://maven.com/parlance-labs/evals/6/join?promoCode=LOOK-AT-YOUR-DATA">with this link</a>.---📚 View <a target="_blank" rel="noopener noreferrer nofollow" href="https://docs.google.com/document/d/1KSUlzN-C8z1q_gHduCi6eKgCzlHVV6u3dT0aypqi_YI/edit?usp=sharing">the syllabus here </a> We've refined this material with over 4,500 engineers and PMs from teams like OpenAI, Google, Meta, Amazon, and Microsoft, folding their feedback in every time. <a target="_blank" rel="noopener noreferrer nofollow" href="https://testimonial.to/ai-evals-course/all">Read what students say →</a>More than a course: an ongoing system, tools, and community for shipping better AI.<ul><li>🎮 A private Discord community for ongoing support, even after the course.</li><li>🤖 6 months of access to our AI Evals assistant.</li><li>♾️ Lifetime access to all recordings, materials, and future cohorts.</li><li>💬 10+ hours of live office hours to get your questions answered.</li></ul>Do you catch yourself asking any of these while building AI applications?<ol><li>How do I test outputs that need subjective judgment?</li><li>If I change the prompt, how do I know I am not breaking something else?</li><li>Where should I focus my efforts? Do I need to test everything?</li><li>What if I have no data or customers? Where do I start?</li><li>What should I measure, and what tools should I use?</li><li>Can I automate evals, and how do I trust it?</li></ol>If so, this is for you. All sessions are live and recorded.📝 We've completely refreshed the material for our Sep 2026 cohort. Scroll down to see the syllabus.

 Build, evaluate, and improve AI agents that work in production.

Build an agent you can measure

Find what's broken with error analysis

Build evaluators you trust

Catch regressions with CI/CD

Red-team for safety before users do

Improve accuracy and cut cost with evidence

Engineers and PMs who ship prompt changes and hope nothing breaks. (You'll learn to measure impact before and after every change.)

Teams still spot-checking AI outputs by hand instead of measuring systematically. (You'll learn how to build automated evals you can trust.)

Leaders who don't know where their AI is failing or where to invest resources. You'll learn how to systematically find &amp; prioritize issues.

Leave with a working agentic analytics system on your machine, a polished analysis, and the patterns and frameworks to make it all yours.

Principal Data Scientist @ Ontra 

Shane Butler

Principal Data Scientist @ Ontra | Ex-Stripe, Nextdoor, PwC, Appfolio

Data Science Leader |Ex-Microsoft

Sravya Madipalli

Data Science Leader@Superhuman (Prev. Grammarly)| Ex-Microsoft, eBay, Nextdoor

Head of Data @ Ontra AI | Ex-LinkedIn

Hai Guan

Head of Data @ Ontra | Ex-Nextdoor, LinkedIn, Pinterest, Meta

Bootcamp 4

Anyone can get AI to produce analysis now. The hard part is knowing whether it is right.This is a five week, hands on course where you build an agentic analytics system in Claude Code, and then do the thing almost nobody teaches: prove it works.Weeks 1 and 2 you build. You set up your own environment, write your first skill, design and build a multi agent system, integrate it into a production repo, and connect it to your team's tools through MCP.Weeks 3 and 4 you make it trustworthy. You measure reliability across repeated runs, triangulate a question across independent methods of analysis, build ground truth, align an LLM judge to human labels, trace every answer back to the query that produced it, and engineer the context that makes the system correct on your company's metrics.Week 5 you put it to work. You swap the model underneath it, cross check one model against another, and turn the system into value your company can see.Ten live sessions, two hours each, with three instructors in the room. Every recording is yours to keep. Free course retake. Private AI Builders alumni community. No coding background required, no Python or SQL is ever written by you.

Build AI Analysts that do the work and prove the numbers are reliable. No code.

Build a multi agent AI analyst system from scratch

Integrate your system into a production analytics repo

Build an eval suite that proves your analyst is right

Align an LLM judge for work that has no right answer

Engineer the context that makes the system correct

Swap to open source models and triangulate across them

Product managers who are tired of waiting in the data team's queue. Stop waiting. Analyze it yourself.

Data scientists who want to multiply their output. You know how to do the analysis manually. Now automate it.

Engineers who want to make data-driven decisions without hiring a data team.

Build AI Agent Skills that Actually Work - 3 Hours Hands-on Workshop

Product Leader and Super IC at Google

Disney+ Hotstar

Zynga

Anshumani Ruddra

Product Super IC at Google with 22+ years crafting experiences and blockbuster products

Cohort 06

Cohort 07

Most product builders use AI agents the same way they used search - typing queries and hoping for the best. That's leaving most of the value on the table.The teams pulling ahead aren't using better prompts. They're teaching their agents. Agent Skills are reusable, testable instruction sets that tell AI agents exactly how to handle specific workflows: your deployment process, your code review standards, your team's edge cases. Write one skill well, and every agent on your team benefits from it, every time, automatically.The problem: nobody is teaching product builders how to create them. Most documentation is written for a narrow technical audience, and the tooling is fragmented across Claude Code, Codex, and Gemini.In this hands-on workshop, you'll go from zero to a working skill installed in your own agent: one that reflects your real workflows, not generic templates. You'll also learn how to evaluate and improve skills systematically, so they get better over time instead of drifting.If you build or are hoping to build products with AI agents: this is the highest-leverage skill you can develop right now.

🤖 Learn to build procedural Agent Skills, transforming your professional mindset from solo prompter to manager of elite AI agents

Shift to an AI Management Mindset

Build Custom, Reusable Agent Skills

Leverage the Open-Source Ecosystem

Evaluate and Iterate for 10x Output

Product Managers: who use Claude Code or Codex daily but rely on ad-hoc prompts - and want repeatable, team-shareable AI workflows.

Entrepreneurs &amp; Founders: building AI-native products who need their agents to encode business logic, not just generate code on demand.

Tech Executives &amp; Product Leaders: who want their teams to stop reinventing prompts and start shipping a shared skills library.

Build robust LLM-powered apps, chatbots, and agents. Learn by writing real code, one line at a time.

CEO @ Actualize

Jay Wengrow

Software Engineer and Educator, Author of A Common-Sense Guide to AI Engineering

Cohort 2

Cohort 3

Do you have a grasp of basic LLM principles but struggle to put it all into code? Tired of scattered documentation, outdated tutorials, and frameworks that impress in demos but collapse in production?You’re not alone. Building with large language models doesn’t have to be confusing - it can be clear, structured, and deeply rewarding.In this course, you’ll build real LLM-powered apps, one line of code at a time. You'll see how an app is built from scratch in real time, and follow along. Along the way, you'll discover not only how to implement these systems, but the why behind each line of code.From chatbots to agents and RAG to evals, you’ll learn the core concepts of building atop LLMs. More importantly, you’ll get how LLMs “think” - allowing you to guide them to do what you want despite their nondeterministic nature. Additionally, you’ll be able to balance quality, latency, and cost, making big-picture decisions about AI-powered app architecture.AI engineering isn’t just another branch of software development - it’s a different mindset altogether.

Gain hands-on AI Engineering proficiency by writing real code

Gain real-world intuition for working with LLMs

Reliably guide LLMs instead of getting unpredictable results.

Build retrieval-augmented assistants that can access proprietary knowledge

Use evals to systematically optimize your LLM app’s performance over time

Assemble agents that act upon the real world

This course is designed for current software engineers who will use Python  to build LLM-powered apps. We won't teach basic coding here.

Software Engineering

We'll be using Python, but we'll keep it simple in case you're more familiar with other coding languages.

Note about Python

If you want an inside look at what's involved with building AI apps but don't plan on coding yourself, you'll still follow what's going on.

Exception: Product Managers

Software engineers who want to build LLM-powered chatbots and agents, or break into AI engineering more generally

Data Scientists/Engineers who want to build their own LLM-powered apps

Product Managers who want to AI engineering literacy to dream bigger and spec sharper

Agentic AI

Coding with AI

AI Workflows

Claude Code

OpenClaw

Vibe Coding

AI Evals

AI Transformation

RAG & Search

AI for PMs

AI for Engineers

AI for Designers

AI for Marketers

AI for Founders

Product Sense

Product Discovery

User Research

Prototyping

Growth

Analytics

Tech Foundations

Strategy

Influence

Leadership

Career Growth

Machine Learning

LLM Ops

Context Eng

Security

System Design

Figma

Design Systems

Visual Design

Design Strategy

Automation

Content Marketing

Demand Gen

Go-to-Market

Product Marketing

Positioning

Social Media

Brand

B2B Marketing

SEO & AEO

AI for Leaders

AI Governance

Communication

Management

People Operations

Exec Presence

Storytelling

Goal-setting

Personal Brand

Fundraising

Everyone

Operators

Data Scientists

Business Analysts

User Researchers

Customer Success

Project Managers

HR Professionals

Sales People

Lawyers

Finance

Investors

Real Estate

Educators

Creators

Engineers and engineering leaders who build, deploy, and scale software systems.

Applying AI tools and techniques to professional workflows — assign alongside more specific AI topics.

Building autonomous AI agent systems that take actions, use tools, and orchestrate multi-step workflows.

Using AI coding assistants and tools to write, review, and ship code faster.

Operating and deploying large language models in production — fine-tuning, inference, monitoring, and cost optimization.