Abi Aryan Inference Infra Queen (GPU Infra & LLM Serving)

I'm from infra background and have learned a lot around internals of AI Inference and then how to design a system on use case basis. In this cohort, we went into very depth of each component of the inference pipeline, understanding how everything works and then utilise the knowledge to design the systems. Abi has been so amazing and have put all the efforts in curating the course, and focuses on everyone's learning and growth. If anyone wants to learn AI Inferencing then this is it, just go for it without second thought. It is way more worth than the price. Thanks a tonne to Abi for creating the course and putting in so much efforts.

Great course! I learned how LLM inference actually works, from KV cache and capacity math to schedulers, routing and admission control. Abi explains the hard parts clearly, and the labs made every concept concrete. For the final project I designed a cluster, built my own control plane and proved every decision on a real GPU. That is where it all clicked. I now understand serving systems instead of just calling an API. Highly recommended.

This is for sure a great way to visit the full spectrum of AI inferencing, from the sizing of clusters and hardware selection to the system design and measurement of all components!

This course is highly valuable for anyone interested in AI Inference Engineering. The course builds up knowledge and concepts from basics of GPU physical limitations to all the way of designing large-scale inference systems for serving models and what to keep in mind designing and operating them. Highly recommend this to anyone without hesitation !

Before this course I was a software engineer building agents, MCP servers and skills on hosted models, without really knowing what happens after a request leaves my code. The course took me through the whole inference path, from the gateway and scheduler through prefill and decode, KV cache, prefix caching and batching. I can now take an open-weight model, work out how much GPU memory its weights, activations and kernels need, and see what's left for the KV cache. From that I can estimate batch sizes, sequence lengths and throughput for my own apps, using each request's shared prefix and its unique tokens. I also learned to build the gateway in front of the models. It sends a request back to the GPU that already holds its KV cache, and it can tell when moving the KV to another worker beats recomputing it. It admits or queues work by priority before the engine runs out of KV, and it scales workers on KV and queue demand rather than GPU utilisation. All of that makes me a better builder of agents as well as a better operator of the models behind them.