Agentic AI Systems: From Prototype to Production
Nick Talwar
CTO | Agentic AI systems, ex-Microsoft
Todd Larsen
CEO, Technical Leaders | ex-Groupon
Building an agent is easy. Building one you can trust in production is not.
Most agents work in the demo. Then they meet real users, real data and real permissions, and they stall. The model is rarely the problem. The trouble is the harness around it, which loses its state in a crash, writes the same record twice, grades its own work, sees credentials it shouldn’t and spends without a cap.
Security, your CTO and your board now ask the same questions of every agent. What can it do unsupervised? What stops it? What does it cost per outcome? How sure are you? “It works” isn’t an answer.
You’ll design, evaluate, contain and correct your own agent, in your own repo. You’ll leave with a hardened, resumable agent, an evals harness that puts an interval on every number, a containment record, and a go/no-go memo reviewed live by peers and by practitioners who ship this work.
Every session pairs a short idea with a build-along on your agent, a small-group teardown of a real case, and a lab with a principal engineer working alongside you. Four labs cover what most courses skip: durable execution, containment, agent identity and cost engineering.
What you’ll learn
Learn to design, evaluate, and ship agentic AI systems that are reliable enough for real products, workflows, and enterprises.
Map its session log, tools, memory, identity, permissions, failure boundary, review points and budget, so you can defend its scope.
Use the 4 Gears to give each step the smallest gear that can own a verifiable result, and cut the steps that don’t need an agent at all.
Harden your riskiest write path and run the loop in a durable runtime. Leave with a harness map, a step map and your checklist score.
Write outcome-graded cases from your own traces, split capability from regression sets, freeze a holdout and plant a canary.
Calibrate an LLM judge against human labels and put a confidence interval on every number, so you know how many cases the call needs.
Set a hard budget cap and read cost per outcome. Leave with an evals harness wired to your agent and results with their intervals.
Run its riskiest tool in a sandbox with default-deny egress, and inject secrets by proxy so the agent never sees a credential.
Give it a revocable identity and a per-step allow-list, pin tool metadata, and drill the kill switch. Leave with a containment record.
Place human review points only where actions are irreversible, and size them so the review burden falls as the evidence builds.
Turn each reviewer fix into an instruction change and a new eval case. Leave with a live review point and a correction log.
Write the trust section security and the board read first: AI bill of materials, permissions, blast radius, re-approval triggers.
Size the resourcing line (reviewer hours, eval upkeep, the budget cap) from your confidence intervals, so the ask matches the call.
Present your memo live to a peer review group and the teaching team, with written feedback following.
Learn directly from Nick & Todd
Nick Talwar
CTO & AI Architect, Founder, ex-Microsoft
Todd Larsen
15+ yrs building teams at Groupon and Digit; helps execs lead AI transformation
Who this course is for
Senior, staff, and forward-deployed engineers who own an agent stuck in pilot and need to harden, evaluate and contain it for production.
Engineering leaders, CTOs and technical founders who read the code and must decide, with evidence, which agent pilots to scale or stop.
Prerequisites
You can clone, install and run a Python project from a terminal
Every session has a lab in your own repo, and the starter kit is Python. The 45-minute pre-work checks this before day one.
Call an LLM API from code and follow an agent’s tool-call loop
We start where the demo ends, so there’s no intro to prompts or agents. You’ll map your own agent’s harness in session 1.
Bring an agent or workflow you want to take to production
Every lab, every eval and your go/no-go memo run on it. Between projects? Build on our runnable sample agent instead.
What's included
Live sessions
Learn directly from Nick Talwar & Todd Larsen in a real-time, interactive format.
Lifetime access
Go back to course content and recordings whenever you need to.
Community of peers
Stay accountable and share insights with like-minded professionals.
Certificate of completion
Share your new skills with your employer or on LinkedIn.
Maven Guarantee
Your purchase is backed by the Maven Guarantee.
Course syllabus
4 live sessions • 8 lessons • 8 projects
Week 1
- 📅
Live Session 1 · Design your agent as a harness, not a prompt
- ✍️
Assignment 1 · Finish your harness map and boundary call
Submit by Jan 21 - 📅
Live Session 2 · Lab: harden the riskiest write path and make it resumable
- ✍️
Assignment 2 · Harden a second path and score your checklist
Submit by Jan 26
Week 1 · Spec: design the harness and make it resumable
Map your agent as a harness, decide which steps need an agent at all, and harden it so a run survives a crash.
Jan
20
Optional: Office Hours
Week 2
Jan
27
Optional: Office Hours
Wed 1/271:00 PM—2:00 PM (EST)Optional- 📅
Session 3 · Lab: build the evals harness on your own agent
- ✍️
Assignment 3 · Grow the golden set and write your first judge
Submit by Jan 28 - 📅
Session 4 · Lab: calibrate the judge, size the evidence, cap the cost
- ✍️
Assignment 4 · Prove it with numbers
Submit by Feb 2
Week 2 · Evals: prove it, calibrate the judge, size the evidence, cap the cost
Build an evals harness on your own agent and put a confidence interval, a case count and a cost cap on every number it reports.
Frequently asked questions
Maven for Teams
Reimbursement
Get your company to pay
Everything L&D needs: email template, receipts, and certificate of completion.
Get reimbursedTeam discount
Learn with your teammates
Save 20%+ when 2 or more teammates enroll in the same cohort.
Save 20%+ with a teamPrivate cohort
Run a cohort for your org
A dedicated cohort with a custom schedule and curriculum, tailored to your team.
Book a private cohort$1,800
USD