Dr. Rajat Dandekar
Free Lesson

How Trillion-Parameter Models Are Served, From Scratch

60 min
Nov 20, 2026 8:30 AM

By continuing, you agree to Maven's Terms and Privacy Policy.

253 students

What you'll learn

Do the arithmetic of serving

Why a 1T model needs 2 TB and 25+ GPUs, why every token reads every weight, and why batching pays for bandwidth.

Tame the KV cache

How MLA, PagedAttention and prefix caching shrink and share the cache, and how 2026 models like Kimi K3 cut it further.

Split, schedule and squeeze a cluster

How TP, EP and pipeline parallelism, prefill/decode disaggregation, batching, FP4 and speculative decoding fit together.

Why this topic matters

Every time you press enter on a frontier model, about 2 TB of weights spread across dozens of GPUs has to produce each new token in roughly 20 ms, for thousands of people at once. That is a memory, networking and scheduling problem more than a math problem. In this free live hour I build up how today's trillion-parameter models are actually served, from one token to a full cluster, with real numbers from DeepSeek, Kimi and vLLM.

You'll learn from

Dr. Rajat Dandekar

Dr. Rajat Dandekar

Vizuara Co-founder | Purdue PhD | Teaching engineers to build AI systems

Education & research
Purdue University
See all products from Rajat
Get free access