
Free Lesson
How Trillion-Parameter Models Are Served, From Scratch
60 min
Nov 20, 2026 8:30 AM
What you'll learn
Do the arithmetic of serving
Why a 1T model needs 2 TB and 25+ GPUs, why every token reads every weight, and why batching pays for bandwidth.
Tame the KV cache
How MLA, PagedAttention and prefix caching shrink and share the cache, and how 2026 models like Kimi K3 cut it further.
Split, schedule and squeeze a cluster
How TP, EP and pipeline parallelism, prefill/decode disaggregation, batching, FP4 and speculative decoding fit together.
Why this topic matters
Every time you press enter on a frontier model, about 2 TB of weights spread across dozens of GPUs has to produce each new token in roughly 20 ms, for thousands of people at once. That is a memory, networking and scheduling problem more than a math problem. In this free live hour I build up how today's trillion-parameter models are actually served, from one token to a full cluster, with real numbers from DeepSeek, Kimi and vLLM.
You'll learn from





Get free access
