Abi Aryan Inference Infra Queen (GPU Infra & LLM Serving)

The great part of this course is the real case studies from multi-tenancy clustering, and how to scaling up the cluster with the good practice. It make me complete the view of how inference company work, which cover from cluster level to kernel engineering. If you are going build the multi-tenacy inference clustering, should take this course.

The course was awesome - learnt so much what happens behind the scenes when we call a chat completions or a LLM. It also gives a sense on how upcoming LLM architecture's are optimizing for memory and latency apart from accuracy. It definitely gives you the edge we don't go deeper into inference in our day to day work

This course was very helpful in giving me a broad, end-to-end understanding of the LLM inference space and connecting the key concepts. I also really liked Abi Aryan’s teaching style. She makes time for questions and prioritizes student understanding over simply covering the material.

This course has been a great learning experience. It goes beyond theory and dives into the practical challenges of building and scaling LLM inference systems, from KV cache management and batching to concurrency, routing, and admission control. The hands-on labs and real-world examples helped me connect the concepts to actual production systems.

I would give this course 10 stars if I could !! Abi is an outstanding instructor. She knows the AI inference stack in depth, both at the level of how the pieces actually work and where the industry is heading. Throughout the cohort she went out of her way to make sure we understood the material. We built every part of the inference stack by hand in this course, so we came out understanding the mechanics, not just the APIs. It is very hands-on. If you work through the homeworks diligently, you will learn more than from any other course I have seen on this topic. The guest sessions with engineers from leading inference companies were a bonus. If you are an engineer in the AI space and want to build your own local or cloud inference stack, or start your own business, I highly recommend this course.