Hugo & Stefan

Cohort 1

Cohort 2

Cohort 3

Cohort 4

Want to build LLM-powered applications but not sure where to start? This handpicked guide features the best resources on Python fundamentals, LLM evaluation, AI agent design, and scaling generative AI systems. Learn from experts like Chip Huyen, Hamel Husain, Shreya Shankar, and Sebastian Raschka to accelerate your LLM development workflow and avoid common pitfalls. Whether you’re an engineer, researcher, or founder, this guide will help you build more reliable and production-ready AI systems.

🚀 Essential LLM & AI Engineering Resources: A Curated Guide for Engineers & Data Scientists

Want to build smarter AI agents? This handpicked guide features the best resources on AI agent design, planning, tool use, and failure modes. Learn from industry leaders like Anthropic, Chip Huyen, and Hugging Face to level up your LLM-powered agents and avoid common pitfalls. Whether you’re an engineer, researcher, or founder, this guide will help you build more reliable and capable AI agents.

Master AI Agents: A Handpicked Guide to the Best Resources

Build LLM-powered software reliably & from first principles. Learn the GenAI software development lifecycle: agents, evals, iteration & more

Hugo Bowne-Anderson

Host of Vanishing Gradients podcast; ex-Outerbounds; expert in DS/ML.

Stefan Krawczyk

Agentforce; Ex-Stitch Fix; 13+ Years building / productionizing data, ML, Agents

Building LLM Applications for Data Scientists and Software Engineers

Staff Software Engineer

Software Engineer

Generative AI Engineer

Data & Infrastructure Manager

Senior Data Scientist

Founder

Data Scientist

Tinkerer

Staff Mobile Engineer

Manager, Metadata Services Unit

If you’re a software engineer or data scientist, chances are you’ve built or seen a proof of concept (POC) for a generative AI app. It’s exciting at first—an impressive demo that showcases the potential of large language models. But here’s the problem: that’s where it usually stops. These POCs often fail to scale into reliable, production-ready applications. The result? Endless iteration cycles, unreliable outputs, and frustration as teams struggle in what we call “POC purgatory.” It doesn’t have to be this way. With the right principles, processes, and workflows, you can build LLM applications that work in production – without overcomplicating pipelines or chasing trends. This course focuses on the iterative development, evaluation, and debugging practices that make AI systems production-ready. We’ll explore tools like RAG, embeddings, and agents, but the emphasis is on building scalable, robust systems – ensuring you leave with the mindset and skills to adapt as the technology evolves. You’ll not only learn how to design, test, and deploy generative AI applications, but also how to iterate and improve rapidly. This isn’t just about building something cool – it’s about creating AI systems that solve real problems, deliver real value, and scale with confidence. What to ExpectThis is a highly interactive course designed to get you results. Across live workshops, discussions, and hands-on projects, you’ll build your skills while solving real-world challenges. •	Live coding sessions to design and refine LLM applications.•	Iteration exercises to explore how small changes can dramatically improve reliability.•	Discussions on user-centric development and integrating feedback loops.•	Expert-led Q&amp;As to help you tackle your toughest challenges. What You’ll Learn •	How to transform POC demos into production-grade applications.•	Strategies for managing non-determinism and optimizing outputs through prompt engineering.•	How to monitor, log, and debug AI systems to ensure reliability and performance.•	Best practices for handling structured outputs and integrating function calling into your applications.•	Building workflows for iterative development and experimentation.•	Practical skills through hands-on app development, including creating a Generative AI app to query PDFs. Bonus Perks All students receive $500 in Modal credits (confirmed). We’re finalizing additional partner credits and will announce them soon. Past cohorts have typically received $1,500+ in total credits; one cohort topped $2,500 when multiple partners stacked. (Actual amounts vary.) - $500 in Modal credits for cloud compute (confirmed).- Additional partner credits: announcing soon.- Past totals: $1,500+ typical; one cohort &gt;$2,500.- Recent partners: Logfire (Pydantic), Google Cloud &amp; Gemini, Hugging Face Pro, Prodigy, Learn Prompting Plus. Guest lectures Learn directly from practitioners shipping AI/LLM systems in production. We’ll announce this cohort’s guest + workshop lineup soon. Enrollment includes on-demand access to all recorded guest talks from prior cohorts (slides/resources where available). Previous guest speakers in previous cohorts include: - Jason Liu (Instructor, 567 Labs)- Shreya Shankar (UC Berkeley, ex-Google Brain)- Zach Mueller (Hugging Face)- Paige Bailey (Google DeepMind)- Ines Montani (Explosion AI, spaCy)- Alan Nichol (Rasa)- Nathan Danielsen (Carvana)- Charles Frye (Modal)- John Berryman (Arcturus, ex-Github)- Skylar Payne (AI executive for startups. Ex-Google. Ex-LinkedIn)- Vincent Warmerdarm (marimo)- Samuel Colvin (Pydantic, Logfire)- Hamel Husain (Parlance Labs) This Course Is For •	Data scientists and machine learning engineers who are sick of unreliable POCs and want to ship reliable LLM applications.•	Software engineers who want to learn how to build generative AI systems and master the LLM software development lifecycle. This Course Is Not For Those •	Looking for an introductory course in AI—some programming knowledge is required.•	Expecting ready-made solutions for AI problems without active participation.•&nbsp;Expecting deep dives into advanced evals, multi-agent systems, or large-scale RAG pipelines. This course emphasizes building and iterating on practical, production-ready systems.•	Unable to commit to hands-on projects, live discussions, and iterative development exercises. Why Take This Course By the end of this course, you’ll: •	Understand the first principles of generative AI development.•	Gain hands-on experience building robust, scalable applications.•	Learn how to troubleshoot, monitor, and optimize AI systems for production.•	Develop workflows to iterate and refine AI applications effectively.•	Leave with the confidence to deploy reliable AI systems at scale.

Master the patterns and processes that underpin successful GenAI app development

Course syllabus

Mondays and Wednesdays

To ensure hands-on practical time, there will be project work to complete throughout the course.

Weekly projects

Course schedule

Data scientists, machine learning engineers who are sick &amp; tired of seeing and building prototypes &amp; want to ship reliable LLM applications

Software engineers who want to learn how to build Generative AI systems and learn the LLM software development lifecycle.

Who is this course for

What you’ll get out of this course

You’ve seen impressive generative AI demos—but how do you take them to production? In this course, you’ll learn the exact workflows, tools, and engineering practices needed to turn concepts into reliable, scalable applications that deliver real value.

Move beyond POCs and build production-ready AI systems

Struggling with inconsistent outputs? Through interactive exercises, you’ll explore proven techniques for optimizing prompts and ensuring your AI delivers actionable results in production settings.

Master prompt engineering for reliable results

When your AI application breaks, you’ll know exactly how to fix it. We’ll walk you through real-world debugging sessions, monitoring techniques, and tools to ensure your systems stay reliable at scale.

 Learn to debug and monitor AI systems like a pro

Want your AI to produce precise, usable results? You’ll learn how to handle structured outputs and function calls, making your applications smarter and more user-friendly.

Build smarter applications with structured outputs

Stop wasting time in endless iteration cycles. Through guided projects, you’ll discover how to set up workflows that refine your applications faster and with less frustration.

Accelerate your development process with iterative workflows

Tired of theoretical knowledge? You’ll build a fully functional app that uses text and image models to query PDFs, giving you practical experience solving complex real-world problems.

Develop hands-on expertise by building GenAI and LLM apps

Whether you’re looking to lead AI projects, impress stakeholders, or level up your technical expertise, this course gives you the tools to stand out as an AI/ML practitioner.

Advance your career with practical, proven techniques

These aren’t just skills for demos—they’re skills for delivering scalable, production-ready systems that solve real-world business challenges.

Solve Real Problems at Scale

Leave the course with a project you can add to your portfolio, demonstrate to your boss, or use to showcase your ability to deploy reliable AI systems.

Showcase Tangible Results

Making LLM Agents Observable & Debuggable

Podcaster, Educator, DS & ML expert

Claire Longo

AI Researcher at Comet | Mathematician | Startup Advisor | ex-Arize AI 📈

LLM agents fail silently, hallucinate, and drift: learn to catch issues early with output checks, trace logs, & metrics.

How to debug and monitor agent behaviour in real-time

Use humans and LLMs to evaluate outputs with real-world workflows and practical examples you can apply immediately.

Work with human annotations and LLM's-as-a-judge

Give your IDE eyes and ears using Opik MCP to add telemetry and metrics, so you can spot and fix AI issues fast.

Using MCPs to level-up your vibe coding with telemetry

Get hands-on examples that work across LLMs and agent frameworks—apply these methods in your stack right away.

Start building today with open-source cookbooks

Don't Tweak Prompts. Engineer Agents.

Skylar Payne

AI made easy. AI executive for startups. Ex-Google. Ex-LinkedIn.

Go from staring at a wall of tokens to systematically finding and fixing bugs with a repeatable 5-step process.

Master the Core Eval Loop

Break down long, chaotic agent traces into clear, gradable checkpoints to instantly see where things go wrong.

Define Actionable Milestones

Use aggregate data to pinpoint the biggest leaks in your agent’s performance and focus your efforts for maximum impact.

Build Failure Funnels

Ship every new version with confidence by tracking task success, regressions, and costs in one holistic view.

Create a Trusted Scoreboard

Context Engineering for AI Agents

John Berryman

AI Consultant and Builder, Co-Author of Prompt Engineering for LLMs, ex-Github

Context engineering = empathizing with AI: shape memory + structure context to guide its “mental state” & boost output.

Understand context engineering as empathy with the model

Unguided agents fail. Reliability comes from user empathy: transparent steps + artifacts that keep humans in control.

Keep agents effective through participatory workflows

Learn to build AI apps that balance empathy for the model and the user to ensure power and trustworthiness.

Design AI applications that balance model and user needs

Vincent Koc

Comet, ex-Microsoft, Qantas

The Hidden Simplicity of GenAI Systems

Learn how four GenAI pillars—instructions, context, reasoning, and tools—support agents, RAG, and workflows.

Understand the core building blocks behind GenAI apps

See how core GenAI ideas combine into popular patterns, & how to move beyond them to design systems that fit your goals.

Reframe complexity into reusable patterns

Walk through two simple but powerful examples so you leave ready to adapt, remix, and build GenAI features your way.

Leave with practical inspiration

10x Your Productivity by Building Personal Agents with MCP

Test prompts, tweak settings, ground in docs, and call APIs. All without wrestling with generic MCP SDKs.

Build Custom AI Tools with FastMCP

Track decisions in agents to debug behavior and guide tool use—no more “Why did it do that?!” moments.

Tame Your Agents: Debugging and Guiding Behavior

Use Mirascope to build agents and re-expose them as tools in your own MCP server, then scale to complex automations.

Develop Scalable Agentic Workflows

Get shippable code and apply it to automate a weekly review, from inbox zero to strategic alignment via your MCP.

Automate Real-World Tasks: Your "Personal MCP" in Action

Why Tool Calling Breaks—and What to Do Instead

Alan Nichol

CTO, Rasa, OSS Sorcerer

Understand the limitations of atomic function calls and ReAct-style orchestration in real-world systems.

Why most “tool use” patterns break at scale

Learn how to guide users through complex, multi-step processes—without relying on brittle prompt chaining.

How to design stateful, multi-turn business logic

See how to combine LLMs and deterministic processes to deliver robust, explainable conversational experiences.

How Process Calling works in production

Discover how structured, reusable business processes reduce flakiness and accelerate iteration.

Debug less and ship faster with modular flows

Build GenAI Systems Fast: AI Studio, Gemini, and Gemma

Ravin Kumar

Google DeepMind, ex-Sweetgreen, SpaceX

Test prompts, adjust model settings, ground your app in documents, and wire up APIs—all from one interface.

Prototype full GenAI systems using AI Studio

Explore retrieval, agentic workflows, and API calls—no boilerplate, just real features built fast.

Build real GenAI features: retrieval, agents, and tools

Learn when to prioritize privacy and control vs. scale—and how to design systems that work for both.

Run locally with Gemma or scale with Gemini

Get working code you can adapt immediately—so you leave ready to ship, not just experiment.

Start building today with reusable code templates

LLMs and Low-Hanging Fruit: Finding GenAI Value Fast

Nathan Danielsen

Builder of Great Products and Engineering Teams

Learn how to turn familiar internal resources into working GenAI prototypes—no new infrastructure required.

Build GenAI apps with your existing internal data

We’ll walk through a repeatable method for connecting your day-to-day workflows to high-impact LLM applications.

Framework for finding GenAI wins in your org

See how to plug GenAI into existing systems and APIs to build something useful in hours, not weeks.

Go from idea to prototype fast—no new stack needed

🛠 Synthetic Data Flywheels: Build Reliable LLM Apps Faster

13+years in MLOps: Ex-Stitch Fix, Ex-Nextdoor, Ex-LinkedIn

Generate structured synthetic data to test edge cases, regressions, and weak spots before users ever see them.

Use synthetic data to catch failures before real users

Create a system that automates testing, validating outputs, and catching failures before deployment.

Build an evaluation harness to test LLM apps pre-launch

Establish a process that continuously refines LLM behavior using systematic evaluations and feedback loops.

Hugo & Stefan

Hugo & Stefan

Courses

Lightning Lessons

Resources

Reviews