Master distributed training and real-world scale techniques from top engineers.

Whether you're an ML engineer looking to move beyond single-GPU experiments, or a product leader seeking to understand the language your AI team speaks, this course will give you the hands-on skills and conceptual clarity to operate confidently at scale.

What Makes This Course Different

This started as a distributed training course. It organically grew into an all-encompassing learning event with world-class speakers who bring deep, real-world experience in modern large-scale training.

The distributed training curriculum remains intact: five hands-on workshops covering today’s core scale-up methods.

Now, each week features hand-tailored guest lectures from top engineers at Hugging Face, Meta, Snowflake, and more.

These expert sessions are aligned to each workshop’s topic, helping you bridge theory with modern production practices.

All course materials and recordings are available after enrollment, and you’ll get free lifetime access to future cohorts.

💡 What’s included:

5 core workshops

15+ guest talks across 3 curated tracks

Weekly live office hours

Class discord with lifetime access

Over $2000+ total credits from Lambda Labs, Modal, Prime Intellect and Hugging Face

100% money-back guarantee (within 30 days of the start of the course)

🏕️ Fireside Chats

Hear from the experts on their real experiences trying to take models to scale, the challenges, and the discoveries

Yuxiang Wei (Meta FAIR)

📣 Conference Talks

Applied Track

Hear how industry leaders are solving real-world scale problems:

Robert Nishihara (Ray, Anyscale): Scaling across thousands of GPUs with Ray

Sami Jaghouar (Prime Intellect): Decentralized global-scale training

Tunji Ruwase (Snowflake): Efficient long-context training with Arctic

Prince Canuma: Local ML workloads using Apple Silicon + MLX

Pretraining Track

Deep dives into LLM pretraining at scale:

Phuc Nguyen (Hugging Face): A practitioner's guide to FP8

Elie Bakouch (Hugging Face): Hyper-optimizing LLMs with MoE, MLA & more

Daniel Han (UnslothAI): Speeding up training with Triton & custom kernels

🧠 Distributed Training Course

Learn the foundations and modern techniques used in real-world LLM scaleups across hundreds or thousands of GPUs.

5 instructor-led workshops:

DDP from scratch and avoiding data bottlenecks

ZeRO (Part 1): How model sharding enables scale

ZeRO (Part 2): Efficiency tradeoffs and stage comparison

Pipeline & Tensor Parallelism: Solving communication slowdowns

Multi-Dimensional Parallelism: Combining all methods for throughput

Guest Lectures include:

Sylvain Gugger (Jane Street): Overview of ZeRO

Wanchao Liang (TorchTitan): DTensor and large-scale pretraining

Wing Lian (Axolotl): 2D Parallelism with Axolotl

Ferdinand Mom (Hugging Face): Multi-dimensional parallelism

Less Wright (Meta): Async TensorParallelism

Matej Sirovatka (Hugging Face): Expert Parallelism for MoE

Marc Sun (Hugging Face): Deployment strategies at scale

🚀 Free Compute & Tools

Get hands-on with real-scale training from Day 1.

We’re proud to be sponsored by:

λ Lambda Labs — $1000 in compute credits

⚙️ Modal — $1000 in compute credits

➡️ Anyscale - $100 in compute credits

🦋 Prime Intellect — $50 in compute credits

✅ Guarantee

If you're not satisfied, we offer a 100% refund up to 14 days after the course begins. No risk, just learning.