Design an Enterprise-Grade Software Factory

Samia Khalid

Principal Applied AI Engineer, Microsoft

Take your agents from prototype to production, shipping 24/7

Your coding agent can open a pull request in twenty minutes. The change still takes days to reach users: a spec, a review of a diff nobody asked for, a flaky test, a sign-off, with you moving work by hand in between. Better prompts, a review bot and more parallel agents each sped up one step, and the wait moved. Nobody can say whether agents save time or just create review work.

The bottleneck is no longer writing code. It's everything around it. Here you learn to design a high-quality agent and the factory around it, so work flows continuously and you step in only where needed: checks someone else can run, evals for every part, reviewer included, a control plane that tracks who owns each job and what's next, and people on the risky changes. Then you make it learn. Retries fix one run. Lessons from run logs, kept as tests, rules and reviewed memory, fix every run after it.

I lead the agentic systems engineers at Microsoft use to ship code and manage incidents, and I'll share the patterns and lessons from running them in production. In the last hour you run a small factory in your own terminal, and leave with a starter kit and a 7-day plan for your team.

What you’ll learn

How to take coding agents from prototype to reliable production: the agent, its evals, the path to users, and a factory that learns.

  • Assemble an agent from model, context, instructions, tools, skills and memory, so every part has one job you can test.

  • Trace a bad result to the part that caused it, so you fix the context or a tool instead of rewording the prompt.

  • Write task briefs that state the goal, the limits, the checks and when to stop, so agents stop guessing what done means.

  • Define done as a check someone else can run, so no agent ever grades its own work

  • Evaluate every agent, the reviewer included, on many real cases and more than once, not one lucky demo.

  • Rerun your evals whenever a model, prompt or skill changes, and catch regressions before they ship.

  • Pair a builder agent with an independent reviewer, and route risky changes like permissions or customer data to a named person.

  • Run every job through a control plane that keeps its state, owner and next step, so work survives a crash or a closed chat.

  • Climb an evidence ladder from CI to shadow runs to staged rollout, and roll back fast when production says no.

  • Turn failures and repeated review comments into tests, rules and reviewed memory, so the next run starts smarter.

  • Mine agent logs for what people keep correcting and what agents keep searching for, not just what crashed.

  • Keep self-learning separate from retries: a retry fixes one run, reviewed memory fixes every run after it.

  • Count accepted work, not tries or pull requests, and price in every failed attempt.

  • Track time, cost and how often people stepped in per job, and find where the factory loses time.

  • Check the worst case, not just the average, before you give an agent more autonomy.

  • Put the day's ideas to work in a hands-on hour, with your own coding agent on a practice repo.

  • Leave with a starter kit, bootstrap template for your own repo, and a 7-day plan to bring one real workflow on your team into a factory.

Workshop agenda

  • [Lecture] Agents, evals, and work you can trust

    Why faster coding hasn't sped up shipping, what every coding agent is made of, and how evals and gates replace the vibe check.

  • [Lecture] From one agent to a factory that learns

    Connect agents from ticket to production, decide where people still review, make the factory learn from its mistakes, and measure whether it pays off.

  • Hands-on lab: Run a small factory in your own terminal

    Put the ideas to work with your own coding agent on a practice repo you clone in class. You leave with a starter kit and a 7-day plan for your team.

Learn directly from Samia

Samia Khalid

Samia Khalid

Principal Engineer, AI-Native Transformation Leader @ Microsoft

See all products from Samia

Who this workshop is for

  • Tech leads and engineers whose agents code fast but whose changes still wait days, and who need a way to ship them safely.

  • Engineering managers with AI-native transformation goals. Decide what to build or buy, and prove it pays off.

  • Engineers preparing for system-design interviews. Design agent systems end to end, with trade-offs you can defend.

Prerequisites

  • Basic hands-on experience with a coding agent

    You've used one on real work, so we skip prompt basics and spend the time on the system around it.

  • A laptop with Git, Python 3.10+ and your coding agent signed in

    You'll git clone the lab and run it on your own account, so setup can't eat the hour. Model usage isn't included.

What's included

Samia Khalid

Live sessions

Learn directly from Samia Khalid in a real-time, interactive format.

The lab kit

The practice repo you clone in class, so you can rerun the lab on your own or walk a colleague through it afterward. And a starter bootstrap prompt for your own repo.

Lifetime access

Go back to course content and recordings whenever you need to.

Community of peers

Stay accountable and share insights with like-minded professionals.

Certificate of completion

Share your new skills with your employer or on LinkedIn.

Maven Guarantee

Your purchase is backed by the Maven Guarantee.

Frequently asked questions

Maven for Teams

Reimbursement

Get your company to pay

Everything L&D needs: email template, receipts, and certificate of completion.

Get reimbursed

Team discount

Learn with your teammates

Save 20%+ when 2 or more teammates enroll in the same cohort.

Save 20%+ with a team

Private cohort

Run a cohort for your org

A dedicated cohort with a custom schedule and curriculum, tailored to your team.

Book a private cohort

$600

USD

Dec 12
·

12–4pm EST

Enroll