Will Brown & Kyle Corbitt

Cohort 1

Gain the key skills for designing effective agents and optimizing their performance. Dive deep into evaluations, tools, MCP, and RL.

Will Brown

Kyle Corbitt

Production-Ready Agent Engineering: From MCP to RL

Digital Ethics 

Business Architect

Machine learning Engineer

developer

Indie Consultant

Principal Machine Learning Engineer

Research Scientist

Senior Data Scientist

Machine Learning Engineer

Modern teams are under pressure to ship LLM agents that actually work in production, but it's difficult to cut through the noise and determine what actually works. There's an ever-growing number of "agent frameworks" promising great results, yet whose abstractions are opaque and difficult to optimize. Blog posts and one-off repos explain pieces of the puzzle, but AI is moving faster than ever. Many engineers struggle to:- Choose the right agent pattern for their use case- Incorporate reliable tool use into agentic workflows.- Evaluate where and why agents fail.- Deploy agents which optimize intelligence, cost, and latency.- Understand when and how to improve agent performance with finetuning and RL. We keep hearing that 2025 is the Year of the Agents. Everyone’s talking about MCP and A2A and GRPO but no one seems to agree on when you should use them. Agentic interactions are becoming table-stakes consumer features, and investors are eager to see that you’re keeping up with the times. Popular agent products like Deep Research, Devin, and Manus are built by companies who don’t want to share their tricks. Open-source alternatives often underperform or are complex to understand and adapt. Textbooks don’t exist yet, and sifting through every new paper is basically a full-time job. The latest API models can make for powerful agents, but costs get out of control quickly. Few people outside of the big AI labs have hands-on expertise in optimizing LLM agents using reinforcement learning. Will and Kyle happen to be two of them. --- What to expect: Beyond core principles, this course emphasizes hands-on practice for building production-ready agents, including:- How to integrate MCP tools for popular services like Notion, Linear, and Slack into your agent applications- How to build your own MCP servers for custom APIs and data- How to scaffold and prompt agents for complex tool workflows- How to evaluate and interactively refine agents with human-in-the-loop prompting- How to use rule-based and LLM-based evaluations as reward signals for RL or synthetic data filtering- How to train cost-effective agents which outperform models like o3 at a fraction of the cost using GRPO The course will have 2x weekly lectures for 3 weeks, and we will have additional sections for office hours (see schedule below). Lecture videos will be available to watch asynchronously, and we'll also have a Discord chat for offline discussions. Lectures will incorporate live coding/prompting with tools like Cursor, Claude Code, and Jupyter notebooks. Familiarity with Python, high-level AI/ML concepts, and LLM APIs is assumed. You will also receive:- $100 in Prime Intellect compute credits- $100 in OpenPipe finetuning credits- 1 year of Weights &amp; Biases Pro ($600 value) --- Course schedule: Lecture 1 (6/17)Agent Patterns and Principles- ReAct, MemGPT, Agentic RAG, Multi-Agent (A2A)- Hands-on demos with HF smolagents + other frameworks Lecture 2 (6/19)Model Context Protocol: When and Why- Client/Server architectures for tool calls- Approaches to auth- Hands-on agentic MCP flow demos with Claude Desktop + Claude Code etc. Lecture 3 (6/24)Evals for Agents- Extending eval techniques to agentic workflows- Rule-based vs LLM-as-judge- Filtering rollouts for synthetic data collection- Brief demo of SFT on filtered rollouts Lecture 4 (6/26)Reinforcement Learning for Busy Engineers- Crash course in RL fundamentals without the math- GRPO vs DPO vs PPO- Demo of GRPO for training a reasoning model (via HF TRL) Lecture 5 (6/24)Formulating Business Problems as RL Tasks- How to think about reward/rubric design for real-world tasks- Environment = Tasks + Tools + Verifiers- Walkthrough of problem formulation for email search (via ART) Lecture 6 (6/24)Training Agents with GRPO- Deep dive into RL experimentation for agent workflows (via ART)- Broader ecosystem: other RL trainers + integrations with existing agent/tool frameworks (smolagents, MCP)

Why should you take this course?

Course syllabus

Tuesdays & Thursdays

2x weekly lectures and at least 1x weekly office hours with instructors

June 17 - July 3

Take-home exercises for more hands-on exposure to the week's topics

Weekly projects

Course schedule

A Senior SWE turned AI Engineer at a Series D SaaS company who's eager to replace brittle pipelines with highly-optimized agents

A Founder + CTO of a Series A startup who wants to offer a best-in-class agentic AI experience to discerning customers

A Technical Director at a Fortune 500 company responsible for evaluating the best approaches and vendors for agentic AI solutions

This is the course for you if you're:

What you’ll get out of this course

Understand key concepts and patterns underlying modern LLM agents, and how to choose the right approach for your use case

Build portable, reliable tools for your agents and data using Model Context Protocol (MCP)

Implement your own Research agents, incorporating custom format instructions and data access

Learn the fundamentals of Reinforcement Learning (RL) and how it applies to agents

Formulate your agentic tasks as RL problems, with evaluation metrics that enable learning from reward feedback

Use RL algorithms like Group-Relative Policy Optimization (GRPO) to train agents which outperform frontier models on your tasks

A holistic understanding of modern principles and techniques for designing production-ready agents and optimizing them with RL

Reward Hacking 101: Keeping Your Agent Honest

Founder at OpenPipe

Students will learn to recognize and diagnose reward hacking in RL systems and in everyday incentive structures.

Spot Reward Hacking Early

Learn to inspect rollout traces with tools like ART or Langfuse to surface and explain unexpected exploit strategies.

Trace-Driven Debugging Skills

Apply quick reward tweaks or auxiliary checks to neutralize hacks and steer models back toward desired objectives.

Fix & Realign Incentives Fast

Training Agents with Reinforcement Learning

Research Lead at Prime Intellect

Overview of RL basics: policies, actions, environments, rewards, how it relates to LLMs via GRPO

A crash course in RL for LLMs

Walkthrough of building a search agent + how to prototype environments via tool integrations + offline evaluations

How to build "RL-ready" agents

Recipes for improving your agent via online RL training with GRPO

Tips and tricks for RL training

Build Your Own AI Research Agent

Researcher at Prime Intellect advancing open-source RL for Agents

Integrate powerful MCP Server search tools into chat and code interfaces

Equip agents with powerful search tools

Convert a single example report into rigorous instructions for your agent to follow on any new request

Create guidelines for search patterns and report style

Learn simple techniques for creating "self-improving" agents that remember your feedback and get better over time

Learn directly from Will Brown & Kyle Corbitt

Will Brown & Kyle Corbitt

Courses

Production-Ready Agent Engineering: From MCP to RL

Lightning Lessons

Reward Hacking 101: Keeping Your Agent Honest

Training Agents with Reinforcement Learning

Build Your Own AI Research Agent