Production-Ready Systems with LLMs and Agents: An Intensive for Engineers

Decide what to hand the LLM and what to keep in deterministic code, the highest-leverage choice in any agent system

Spot the tasks where a model adds risk without adding value, and replace them with plain logic.

Design prompts and tool interfaces as narrow contracts, so the model's job stays small and testable.

Set a per-request cost and latency budget, then design the system to live within it

Apply caching, batching, and model routing to cut spend without losing quality

Right-size the model per task instead of defaulting to the biggest one everywhere

Build retries, timeouts, and fallbacks so a slow or failing model never stalls the system

dd graceful degradation paths for when the model is wrong, unsure, or unavailable

Contain non-determinism with validation and guardrails before output reaches a user

Know when to use tool calls, planning loops, memory, or multiple coordinated agents.

Recognize when a single well-scoped agent beats a complex multi-agent design

Map each pattern to its failure modes so you choose with eyes open, not by hype

Build an eval harness that scores changes before they ship, not after users complain

Combine offline test sets with online signals to catch drift and silent regressions

Turn a vague "it feels worse" into measurable quality gates in your pipeline

Trace every LLM call so you can debug what the system did, not what you assumed

Watch cost, latency, and quality on dashboards that surface problems early

Place human oversight and prompt versioning where they actually reduce risk