This course is popular
19 people enrolled last week.
From AI Engineer to AI Inference Engineer
If you’re an AI/ML engineer who can already prompt or fine-tune models but you’ve never been able to answer questions like:
- Why is my 70B model using 120 GB of VRAM and still slow?
- How do I serve 500 concurrent users on 4xH100s without going broke?
- What actually happens inside FlashAttention / PagedAttention / tensor parallelism?
- How do I save my company millions running open models in production?
… then this is the course you’ve been waiting for.
A bit of content update is - 𝐭𝐡𝐞𝐫𝐞 𝐢𝐬 𝐧𝐞𝐰 𝐩𝐥𝐚𝐭𝐟𝐨𝐫𝐦 𝐞𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠, 𝐢𝐧𝐟𝐫𝐚 (+𝐜𝐥𝐮𝐬𝐭𝐞𝐫) 𝐝𝐞𝐬𝐢𝐠𝐧 𝐚𝐧𝐝 𝐝𝐢𝐬𝐭𝐫𝐢𝐛𝐮𝐭𝐞𝐝 𝐬𝐲𝐬𝐭𝐞𝐦𝐬 𝐦𝐚𝐭𝐞𝐫𝐢𝐚𝐥 𝐠𝐞𝐭𝐭𝐢𝐧𝐠 𝐚𝐝𝐝𝐞𝐝 for Cohort 2.
What you’ll learn
Most engineers load models with Hugging Face then serve something basic and get stuck on cost, latency, or scale. In this course, we will
Learn to serve a tiny model (e.g. Phi-3-mini or TinyLlama) on a single consumer GPU in <15 minutes.
Learn how to develop and deploy inference gateways, do endpoint management and learn steaming vs non-streaming architectures
You'll learn concrete request‑to‑token mental model before hardware and optimization detail.
Use real tools to see exactly where time/memory is wasted.
You'll learn to calculate exactly how many tokens, bytes and VRAM your model will take.
You'll learn how to build an inference engine from scratch and the internals of vLLM, SGLang in detail and an overview of the rest..
You'll learn how to optimize for compute versus memory.
We will implement compute management tricks and then memory management optimizations incl KV Cache, Quantization etc.
How to decide which hardware to use, for which model and what will scale and where will different model/hardware combinations hit bottleneck
Scaling models doesn't just mean throwing more compute at the problem
You'll learn distributed systems architecture design using Ray and Kubernetes - Fleets, Multi-Node and Multi-GPU & how to scale inferencing
We will implement distributed inferencing for 70B–405B models.
The internals- what happens when your system gets a request, how tokenization, batching, memory allocation is done on your GPUs.
Inference Optimization techniques (speculative decoding, chunked prefill etc) both at prefill & decode stages for dense, sparse, and MoE
How to do concurrency management at request/model/hardware level. How to use Load balancing and parallelism (Data, Tensor, Model, Expert).
We have a huge lineup this time around - to be announced soon.
Learn directly from Abi
Abi Aryan
Founder and Research Engineering Lead @ Abide AI
Who this course is for
AI engineers, ML infrastructure engineers, and backend developers who own inference cost and need to 10–50× optimize it.
Founders & engineers building LLM apps who are tired of burning money on OpenAI
AI engineers who don't understand system design for building reliable applications
Anyone on job market for Inferencing/Solns Architect role
Prerequisites
You have shipped at least one non-trivial Python project
We cannot teach you the basics of Python code.
You can already load and run an LLM using HF Transformers
If you have never touched AI models, this course is not for you.
Comfortable with basic terminal/SSH, git, yaml & json files
We can help with Docker and Kubernetes, don't worry.
What's included
Live sessions
Learn directly from Abi Aryan in a real-time, interactive format.
Lifetime access
Anyone who signs up for the course will have lifetime access to the course and can audit recording/materials too for the next cohort as I do update my course to always keep you ahead of the curve.
Resume Review
We will have 1-on-1 resume review if you are currently on the job market or looking for a career transition
171-page system design interview guide
It covers 150 practice questions covering interview questions for 1. Systems Architecture Design 2. Inference Optimization & Serving 3. RAG & Context Engineering 4. Reliability and Guardrails 5. Observability, Evaluation & Monitoring, and finally 6. Tradeoffs, Scenarios & Integrations
Community of peers
A discord channel to stay connected with peers.
Guest Speakers
Learn from industry professionals and their experiences.
Certificate of completion
Share your new skills with your employer or on LinkedIn.
Maven Guarantee
Your purchase is backed by the Maven Guarantee.
Course syllabus
37 lessons • 2 projects
Week 1
Free resources
Schedule
Live sessions
2-3 hrs / week
We meet every Saturday 4-6/6.30 pm CET giving you one clear Sunday to rest and recuperate to be ready for the work-week on Monday. This is the key to last 2 months without burning out.
Weekly Project
1-3 hrs / week
Abi will be available to help you in case you get stuck. And you will be discord helping debug each other's solutions - helps learn how people from different backgrounds and different industries approach the problem differently.
Quizzes and Async content
1 hr / week
This was the most loved part of Cohort 1. Every week you will get written notes on the class material and a quiz to test your understanding - the answer key will also be shared. No judgements whatsoever.
Frequently asked questions
Previously Sponsored (Cohort 1) By
This cohort's sponsors be will be shared once we have a good class size estimate.
Maven for Teams
Reimbursement
Get your company to pay
Everything L&D needs: email template, receipts, and certificate of completion.
Get reimbursedPrivate cohort
Run a cohort for your org
A dedicated cohort with a custom schedule and curriculum, tailored to your team.
Book a private cohort$2,499
USD