Agent Engineering: Build, Evaluate, and Deploy AI Agents
Stefan Jansen
Author, ML for Trading · Applied AI
From agent fundamentals to a live multi-agent system
Understand how AI agents reason, use tools, manage information, and collaborate. Learn how to implement these systems in Python and take them through evaluation and deployment.
We cover agent fundamentals and architectural patterns, including ReAct and Reflexion; tools, context, memory, and state; multi-agent coordination; and evaluation, monitoring, and production operation. You'll work with direct API implementations and compare approaches using LangGraph and CrewAI.
Our worked application is based on Bridgewater's AIA Forecaster. Forecasting gives us measurable outcomes for comparing design choices, and the reference implementation runs live on my website.
You leave with a working system you understand and can modify, plus concepts and patterns you can apply to research agents in other domains.
What you’ll learn
Understand the design choices behind AI agents and apply them in a working system, from the agent loop to deployment.
Understand the ReAct loop and compare patterns such as Reflexion and Tree of Thoughts.
Identify how prompts, tools, context, and execution loops shape an agent's behavior.
Follow direct API implementations and compare them with LangGraph and CrewAI.
Run and modify the working code to understand what each implementation controls.
Connect data sources, define tool interfaces, and add reusable skills.
Understand how agents retain state and select information for each step.
Combine independent research agents through aggregation and supervisor reconciliation.
Compare collaboration patterns and test whether additional agents improve results.
Use traces and ablations to connect design changes to their effects.
Score forecasts on resolved questions and assess calibration and accuracy.
Follow the live deployment and understand scheduling, persistence, monitoring, and run controls.
Run the daily forecasting workflow and extend it with your own question or market.
Workshop agenda
[Lecture] Agent fundamentals and design patterns
Understand agent loops, tools, and architectural choices, including ReAct and Reflexion. Introduce the forecasting application based on Bridgewater's AIA Forecaster.
[Hands-on] Run and modify a research agent
Run the reference implementation, inspect the ReAct loop, and change the system prompt. Compare how the agent reasons and uses tools.
[Lecture] Context, tools, memory, and state
Design tool interfaces and choose what information the agent needs. Understand working state, persistence, skills, and execution controls.
[Hands-on] Add a data source and a skill
Extend the agent with a data source and a custom skill. Check what information reaches the model and how it affects the research.
[Lecture] Multi-agent architectures and coordination
Compare independent research, debate, aggregation, and supervisor reconciliation. Examine when collaboration helps and what additional complexity it introduces.
[Hands-on] Compare pipeline configurations
Change agent count, aggregation, and supervisor settings. Run the same questions across configurations and compare results.
[Lecture] Evaluation and calibration
Choose metrics and evaluation data. Cover Brier scores, calibration, ablations, and information leakage when testing a forecasting agent.
[Hands-on] Score and evaluate forecasts
Score forecasts against resolved questions, read calibration curves, and compare configurations on the same dataset.
[Lecture] Frameworks and production deployment
Compare direct Python API implementations with LangGraph and CrewAI. Follow the live system's data flow, scheduling, monitoring, storage, and deployment.
[Hands-on] Run the daily forecasting workflow
Configure a data connector, run the pull, forecast, and track cycle, and extend the daily runner with a question or market of your own.
Learn directly from Stefan
Stefan Jansen
Author, ML for Trading · Founder, Applied AI
Who this workshop is for
Quant researchers and data scientists who want to build agents for forecasting, financial research, and evidence-based analysis.
Python developers and ML engineers who want to understand agent architecture and adapt working implementations to their own projects.
Technical leads who want to make informed choices about agent design, frameworks, evaluation, and deployment.
What's included
Live sessions
Learn directly from Stefan Jansen in a real-time, interactive format.
5.5 hours of live teaching and hands-on work
Work through the concepts with Stefan, then run, modify, and compare implementations in guided exercises. Ask questions as you connect each concept to the forecasting system.
A working agent system you can extend
Keep the forecasting repository, CLI, dashboard, and configuration profiles. Add data sources, skills, or questions and use the system as a starting point for your own research agents.
11 notebooks covering the engineering concepts
Explore agent loops, tools, context, state and memory, coordination, evaluation, and framework comparisons. The notebooks accompany Chapter 24 of Machine Learning for Trading.
Maven Guarantee
Your purchase is backed by the Maven Guarantee.
Frequently asked questions
Maven for Teams
Reimbursement
Get your company to pay
Everything L&D needs: email template, receipts, and certificate of completion.
Get reimbursedTeam discount
Learn with your teammates
Save 20%+ when 2 or more teammates enroll in the same cohort.
Save 20%+ with a teamPrivate cohort
Run a cohort for your org
A dedicated cohort with a custom schedule and curriculum, tailored to your team.
Book a private cohort$649
USD
10am–3:30pm EST