Mark Kim-Huang

AI Agents & Multimodal LLMs

Co-Founder & Chief Architect at Gradient

Determine when you should consider using adapted models for table reasoning, pdf parsing or sentiment analysis.

Why and When to Use Task Adapted Models

Use low code APIs to train a base model for a specific task.

Fine-tune LLMs for Task Adaptation

Learn about how task-adapted LLMs can help move your business closer towards deploying autonomous assistants.

How to Shift Towards Agentic Workflows

👋 Welcome to your next great LLM adventure! 🏗️ 🚢🚀 We invite you to join the close-knit AI Makerspace community and start building, shipping, and sharing with us! 🤩 Each cohort culminates in a unique live Demo Day experience! 🧑‍💻 You will learn to prototype LLM applications according to best practices, using the four primary patterns of Generative AI as first principles: Prompt Engineering, Retrieval Augmented Generation (RAG), Fine-Tuning (of LLMs and embedding models), and Agents. 🧑‍🤝‍🧑 You will get hands-on building and sharing complex LLM applications weekly, receiving constant feedback during live sessions and through your homework submissions via instructors, peer supporters, and your fellow cohort members. 📇 RAG systems are front and center in the course. You'll learn how to build applications from quick prototypes with OpenAI GPT models to scalable production applications with the latest open-source LLMs and embedding models. ☁️ While never getting locked into a single cloud-computing vendor, you'll orchestrate systems from scratch in Python and using core version-controlled LangChain constructs, the leading framework for building production LLM applications. 🧐 You will outfit your applications with visibility and monitoring tooling from LangSmith, and you'll learn to evaluate your applications' inputs, outputs, and usage! 🚀 Optimize your prompting with DSPy, your RAG applications with advanced retrieval and chunking, and your LLM inference and serving with AWQ and vLLM! -- **NOTES** **💰 How much do the tools cost? Don't worry! This course is for the GPU-poor among us! This course adapts to industry-standard open-source software and compute hardware and is cloud platform agnostic. We strive to keep your additional costs low (below ~$100 total). Check Q&amp;A for more info!** **🙋 What are the prerequisites? All applicants must complete THE AI ENGINEERING BOOTCAMP CHALLENGE. This will require you to set up your LLM application development environment and deploy your first LLM application! If you're unsure if this could be a fit for you, submit your application and give the challenge a try! It is the best way to test whether or not this level of coding and detail is what you're looking for**

Prototype LLM apps and get them into Production.  Ship fast, then scale.

Provide a brief description for each module as a way to sell the value of what students will be learning throughout the course.

First Module

Second Module

Third Module

Course syllabus

<ul><li>W1: Jan 14th, 16th</li><li>W2: Jan 21st, 23rd</li><li>W3: Jan 28th, 30th</li><li>W4: Feb 4th, 6th</li><li>W5: Feb 11th, 13th</li><li>W6: Feb 18th, 20th</li><li>W7: Feb 25th, 27th</li><li>W8: Mar 4th, 6th</li><li>W9: Mar 11th, 13th</li><li>W10: Mar 18th, 20th</li></ul> * All sessions are recorded!

Tuesdays & Thursdays

Each class period, we will get hands-on with Python coding homework!

Weekly Programming Projects

Open to the public - Friends, family, coworkers, and bosses welcome! <a href="https://www.youtube.com/playlist?list=PLrSHiQgy4VjGf2r-0n__jMDYtiLOf6OPo" rel="noopener noreferrer" target="_blank">AIE1 Demo Day Playlist</a> &amp; <a href="https://www.linkedin.com/events/7177388473547468800/comments/" rel="noopener noreferrer" target="_blank">LinkedIn Event</a><a href="https://www.youtube.com/playlist?list=PLrSHiQgy4VjEmTW_5SYdSJT8xOKMtJC5b" rel="noopener noreferrer" target="_blank">AIE2 Demo Day Playlist</a> &amp; <a href="https://www.linkedin.com/events/7194472892556853248/comments/" rel="noopener noreferrer" target="_blank">LinkedIn Event</a><a href="https://www.youtube.com/playlist?list=PLrSHiQgy4VjEkgERc9palQ4W_LTM22UyN" rel="noopener noreferrer" target="_blank">AIE3 Demo Day Playlist</a> &amp; <a href="https://www.linkedin.com/events/7219879535939330049/comments/" rel="noopener noreferrer" target="_blank">LinkedIn Event</a>

Demo Day!

Dr. Greg: Thursdays @ 8 AM PTThe Wiz 🪄: Fridays @ 3 PM PT

Instructor Office Hours

We typically cover 6-7 days per week with 1 or more office hours held by peer supporters!

Peer Supporter Office Hours

Tuesdays at 9 AM PT: For International (EMEA) students, a Certified AI Engineer will hold weekly 60-minute deep dive sessions!

International-Friendly Weekly Deep Dive Sessions

Course schedule

Data scientists and machine learning engineers aiming at becoming full-stack AI practitioners.

Software engineers who want to build complex LLM applications.

Technical product managers who guide their teams on what to build, why, and how.

Who is this course for

What you’ll get out of this course

Student success will help you achieve your goals before and after certification!

🤝 1:1 Career Coaching

Certified AI Engineers get direct access to job opportunities from our network

💼 Hiring Opportunities

Work with certified AI Engineers in small groups to discuss and code every class!

🧑‍🤝‍🧑 Peer-Supported Live Coding

You will present a unique project live to a cohort of your peers and the public!

🤩 Demo Day!

Build Retrieval Augmented Generation applications and ground LLM outputs in your own reference data!

🙋 Retrieval Augmented Generation (RAG)

Build complex LLM applications capable of reasoning, action, and working with external tools!

🕴️ Agents

Leverage in-context learning as an engineer for prototyping, RAG applications, agents, and more!

🧑‍💻 Prompt Engineering

Adapt open-source embedding models to your domain-specific language or data!

🔢 Embedding Fine-Tuning

Efficiently train LLMs to perform specific tasks using techniques like PEFT-LoRA and quantization!

⚖️ LLM Fine-Tuning

Instrument LLM applications for quantitative metrics-driven development!

📈 Evaluation

Debug, test, and monitor applications built using any leading LLM framework!

🔎 Visibility

Ensure efficient and scalable inference using the right tools and techniques!

🚀 Inference & Serving

Build RAG & Agents w/ LangChain.  Fine-Tune Embeddings & Llama 3.1.  Deploy to Prod in the Cloud & On-Prem.  Monitor and Evaluate w/ LLM Ops

AIE4

"Dr. Greg" Loughnane

Co-Founder & CEO @ AI Makerspace

Chris "The Wiz 🪄" Alexiuk

Co-Founder & CTO @ AI Makerspace

AIE3

AIE5

🧑‍💻 LLM Engineering refers to the evolving set of best practices for training, fine-tuning, and aligning LLMs to optimize their use and function. 🤖 Whether you’re looking at OpenAI’s GPT series, Google Gemini, Anthropic Claude, Mistral, Grok, or any other model provider, the core underlying architectures are extremely similar, from training methods to the data they use. Small Language Models (SLMs) with parameter counts under 7B, like Phi-3 from Microsoft, also have the same architecture patterns. 🏫 This course will provide you with the foundational concepts and code to train, fine-tune, and align LLMs using industry-standard and emerging approaches from the open-source edge heading into 2025. 🤓 Become the expert in your organization on all things training (pretraining, post-training), fine-tuning (supervised fine-tuning, instruction tuning, chat tuning, etc.), alignment (PPO, DPO, etc.), Small Language Models (SLMs), Model Merging, and more!

Understand the inner workings of LLMs from first principles

<ul><li>LLME&nbsp;vs. MLE</li><li>Best-practice tokenization strategies</li><li>Understanding the role of embeddings</li><li>Transformers = Attention Is All You Need</li><li>Encoder/Decoder (BART-style)</li><li>Encoder only (BERT-style)</li><li>Decoder only (GPT-style)</li><li>Build encoder-decoder and decoder-only models from scratch</li></ul>

LLM Engineering Foundations

<ul><li>GPT: "Semi-supervised sequence learning"</li><li>GPT-2: "Zero-shot task transfer"</li><li>GPT-3: "In-context learning"</li><li>Understand datasets and benchmarks for training and tuning</li><li>Compare model architecture, training, scale, limitations, and impact</li><li>Implement unsupervised pretraining, supervised fine-tuning, and instruction-tuning</li></ul>

LLM Training & Fine-Tuning

<ul><li>InstructGPT: "Aligning LLMs with human intent"</li><li>Reference (supervised fine-tuned), Policy (target), and Reward models </li><li>Proximal policy optimization (PPO) algorithm</li><li>Reinforcement Learning with Human Feedback (RLHF)</li><li>Implement RLHF and RLAIF</li><li>Limitations and broader impacts</li></ul>

LLM Alignment

Class!

Office Hours

Peers supporters are here to serve as your destination group; they're available for homework help, concept deep dives, and to hang out with! Find them live in class, on Discord, and even in their own dedicated weekly sessions!

Peer Support Sessions

Data scientists and Machine Learning Engineers who want to train, tune, and align LLMs and SLMs

Software Engineers who want to learn the fundamentals of Machine Learning and LLMs (because they have to).

Aspiring AI Engineering Leaders interested in training and deploying proprietary LLMs and applications

By the end of this course, you will be able to:

🦾 Build, train, and evaluate Language Models using transformer variants (GPT, BERT, BART)

🧐  Calculate self-attention and understand the latest implementations (Flash Attention, FA2)

🔠 Demystify embedding layers, embedding representations, pre-trained vs. learned embeddings, ROPE, and more!

🪙 Decode embedding space representations for optimal next-token prediction

🔡 Build, train, and evaluate embedding models (like those in 🤗 Sentence Transformers)

🚇 Complete unsupervised and continued pretraining of LLMs and SLMs from scratch

🚉 Fine-tune pre-trained LLMs for instruction-following, chat, and more via parameter-efficient methods

🛤️ Align LLMs to balance helpfulness with harmlessness and other criteria (RLXF, DPO)

🚀 Explore frontiers of LLM Engineering (SLMs, MOEs, Merging, alternative fine-tuning, and more!)

Master LLMs and embedding models through transformers, embeddings, next-token prediction, pretraining, fine-tuning, and alignment!

Cohort 2

Cohort 3

Multi-agent LLM systems, while powerful, face hurdles. Optimizing teamwork, managing complex information flow, and ensuring security are just a few. This course equips you to grasp the core principles of multi-agent LLMs, identify use cases, and explore how to integrate them into current products. By the end, you'll be well on your way to becoming a multi-agent LLM system architect, able to design and build the future of multi-player human-machine systems.

Become a Multi-agent LLM Systems Architect

We will have group discussions and guests lecturers on Thursdays, and project office hours on Sundays.

Thursdays & Sundays

Course will run for 6 weeks in September and October. You will have access to reading / watching material, instructor, and hands-on teaching assistants.

Sep. 9, 2024 - Oct. 25, 204

There will be a project that you will do alongside the course material with weekly milestones.

Weekly projects

You're a product person who has experimented with LLM APIs like OpenAI or open source LLMs locally, OR created custom GPT agents.

You're a machine learning person looking to translate LLM potential into practical architectures, strategies and products.

You are software person who cares about the reliability &amp; robustness of the LLM systems you pitch to stakeholders (investors, execs).

This course is for you if...

At the end of this course you will ...

Grasp how multiple LLMs can collaborate with each other, get feedback from or assign tasks to humans, ad use tools to tackle complex problems

Understand the core principles of multi-agent LLM systems

Bridge the gap by crafting strategies to seamlessly integrate LLMs with existing products, boosting user experience, productivity, and business metrics.

Integrate LLMs with existing product offerings to enhance user and customer experience

Learn what it takes to build robust multi-agent LLMs. Understand security best practices and ensure system reliability.

Understand key considerations for securing, robust, and reliable LLM systems

Learn to design, implement, and deploy near-term multi-agent systems to solve business problems.

Amir Feizpour

10 yrs in NLP, Founder of AISC - Community of 5k AI devs and founders

Cohort 2409

Cohort-2501

A properly fine-tuned LLM can achieve performance that outshines leading foundation models. However, achieving these results is not as simple as making an API call. In this course, we'll demystify fine-tuning by systematically going step-by-step through the entire process. Step 1: Curate DataStep 2: Train ModelStep 3: Evaluate ResultsStep 4: Apply GuardrailsStep 5: Deploy ModelStep 6: Improve Results For each step, I will provide proven strategies and real-world case studies from my experience as an AI consultant. When it comes to fine-tuning, the devil is in the details. Practitioners must be precise and exacting to reach peak performance. I will provide hands-on help so that students successfully build an end-to-end fine-tuning solution during the course project.

Fine-tuning that *actually* works

1-2 modules per week with lessons walking you through the step-by-step process of fine-tuning an LLM. Each module includes recorded demo videos, coding examples, and additional resources to help you complete the next project.

Self-Paced Lessons

Students will design and implement a fine-tuned model of their choosing. Each week, students will apply strategies from the lessons to further improve their model.

Course Project

Live Q&amp;A calls with Scott and the rest of the cohort to answer questions and receive hands-on help with that week's projects.

Q&A with Scott

Each student will receive a live coaching session with Scott to review their project, receive personalized feedback, and discuss how to improve their AI skills further.

Private Coaching Session

Software engineers and data scientists who have dabbled with LLMs and are considering a career switch to AI.

AI and ML engineers who want to enhance their skillset.

Startups who are evaluating whether fine-tuning would be beneficial for their company.

Create a fine-tuning dataset that is ideal for your use case. Adjust your dataset to handle nuances and edge cases.

Curate High-Quality Data

Apply the QLoRA algorithm to fine-tune a model (and understand what's actually happening under the hood!). Conduct hyperparameter tuning to optimize training for your use case.

Train a Model

Design a test suite to systematically evaluate model results. Identify weaknesses in your model and determine how to fix them.

Evaluate Results

Deploy your model using techniques such as quantization to reduce costs. Optimize inference parameters for your use case.

Optimize Costs and Scale

Analyze the production impact of your model. Implement a process to continuously improve results.

Build a Data Flywheel

Identify the pros and cons of various open-source LLMs and evaluate how their performance compares to the leading proprietary models. Analyze the current state of LLMs and where they are likely to go in the future.

Explore Open-Source LLMs

Achieve optimal performance from your fine-tuned LLM.

Lightning Lessons

AI Agents & Multimodal LLMs