Cohort 2

Master language models and embedding models through training, fine-tuning, aligning, distilling, and merging transformer architectures!

"Dr. Greg" Loughnane

Co-Founder & CEO @ AI Makerspace

Chris "The Wiz 🪄" Alexiuk

Co-Founder & CTO @ AI Makerspace

LLM Engineering - Foundations to SLMs

🧑‍💻 Language Model Engineering refers to the evolving set of best practices for training, fine-tuning, and aligning LLMs to optimize their use and function to balance performance with efficiency. These best practices have formed the basis for the LLMs, or Large Language Models (a.k.a. Foundation Models) and Small Language Models (SLMs) of today. 🤖 Whether you’re looking at OpenAI’s GPT series, Anthropic Claude, Grok, Google Gemini, Mistral, or any other model provider, the core underlying transformer architectures are similar, as are the training and tuning methods. When chasing after high scores on LLM benchmarks or creating state-of-the-art SLMs, techniques like Model Merging and Distillation are important as well. 🏫 This course will provide you with the foundational concepts and code to train, fine-tune, and align state-of-the-art SLMs and LLMs using industry-standard and emerging approaches from the open-source edge heading into 2025. 🤓 Become the expert in your organization on all things training (pretraining, post-training), fine-tuning (supervised fine-tuning, instruction tuning, chat tuning, etc.), alignment (PPO, DPO, etc.), Small Language Models (SLMs), Model Merging, Distillation, and more!

Understand LLMs from first principles; build custom Small Language Models (SLMs)

Data scientists, researchers, or engineers experienced with classic Machine Learning who want to build custom Small Language Models (SLMs).

Product managers who need to understand the core concepts and code behind custom SLMs to effectively lead their product teams.

Stakeholders who aim to train, fine-tune, or align proprietary SLMs for their customers in 2025.

Who is this course for

By the end of this course, you will be able to:

🦾 Build, train, and evaluate Language Models using transformer variants (GPT, BERT, BART)

🧐  Calculate self-attention and understand the latest implementations (Flash Attention, FA2)

🔠 Demystify embedding layers, embedding representations, pre-trained vs. learned embeddings, ROPE, and more!

🪙 Decode embedding space representations for optimal next-token prediction

🔡 Build, train, and evaluate embedding models (like those in 🤗 Sentence Transformers)

🚇 Complete unsupervised and continued pretraining of LLMs and SLMs from scratch

🚉 Fine-tune pre-trained LMs for instruction-following, chat, and more via parameter-efficient methods

🛤️ Align LMs to balance helpfulness with harmlessness and other criteria (RLXF, DPO)

🚀 Explore frontiers of Language Models (Mixture-of- approaches, Model Merging, alternative fine-tuning, and more!)

<ul><li>LLME&nbsp;vs. MLE</li><li>Best-practice tokenization strategies</li><li>Understanding the role of embeddings</li><li>Transformers = Attention Is All You Need</li><li>Encoder/Decoder (BART-style)</li><li>Encoder only (BERT-style)</li><li>Decoder only (GPT-style)</li><li>Build encoder-decoder and decoder-only models from scratch</li></ul>

LLM Engineering Foundations

<ul><li>GPT: "Semi-supervised sequence learning"</li><li>GPT-2: "Zero-shot task transfer"</li><li>GPT-3: "In-context learning"</li><li>Understand datasets and benchmarks for training and tuning</li><li>Compare model architecture, training, scale, limitations, and impact</li><li>Implement unsupervised pretraining, supervised fine-tuning, and instruction-tuning</li></ul>

LLM Training & Fine-Tuning

<ul><li>InstructGPT: "Aligning LLMs with human intent"</li><li>Reference (supervised fine-tuned), Policy (target), and Reward models </li><li>Proximal policy optimization (PPO) algorithm</li><li>Reinforcement Learning with Human Feedback (RLHF)</li><li>Implement RLHF and RLAIF</li><li>Limitations and broader impacts</li></ul>

LLM Alignment

Course syllabus

LLM Engineering vs. AI Engineering: What's the Difference?

Prerequisites

Understanding supervised learning, unsupervised learning, and neural network architectures is required. Introductory NLP and Computer Vision knowledge is encouraged. Not sure where to start? Read <a href="https://info.deeplearning.ai/machine-learning-yearning-book" rel="noopener noreferrer" target="_blank">this</a>.

A background in fundamental Machine Learning and Deep Learning

Understand basic Python syntax and constructs. You should be comfortable training and evaluating simple ML &amp; DL models using test, train, and dev sets. Not sure where to start? Look <a href="https://www.deeplearning.ai/short-courses/ai-python-for-beginners/" rel="noopener noreferrer" target="_blank">here</a>.

A ability to program in Python within a Jupyter Notebook Environment

Deep Learning Developer Relations Manager

What people are saying

Meet your instructors

Class!

Each class period, we will get hands-on with Python coding homework!

Weekly Programming Projects

Dr. Greg: Thursdays @ 8 AM PTThe Wiz 🪄: Fridays @ 3 PM PT

Office Hours

Peers supporters are here to serve as your destination group; they're available for homework help, concept deep dives, and to hang out with! Find them live in class, on Discord, and even in their own dedicated weekly sessions!

Peer Support Sessions

Course schedule

We teach concepts AND code. Never one or the other. AI has accelerated so quickly that anyone who's been a manager the last few years has not yet seen the code for themselves.

Hands-on learning at the LLM edge

Grow your network. Build, ship, and share with a community! Build relationships with peers and instructional staff to unlock opportunities in the years ahead.

Pair programming made fun and easy

Join a community of like-minded AI practitioners who are all in on Generative AI, and who are heading down the same career path as you are.

Course completed by

About this course

Get notified about the next cohort

Learn from live sessions