Where the Data Can't Leave: Self-Hosted AI for Regulated Industries

Thomas Underhill

I build AI that has to survive an audit

Compliance did not say no to AI. It said no to one deployment model.

Your team built something that worked. Then privacy or compliance looked at where the data would go, and it stopped. That was a year ago, maybe two, and it now feels settled rather than pending.

Two things have changed since that decision.

Models small enough to self-host got good. Two years ago the honest answer to "can we run something useful ourselves" was no. Today a quantized model on a laptop does work that needed a frontier API. The gap has not closed, and I will show you where it still is, but it is no longer the calculation you ran.

And waiting is not free. The alternative to a sanctioned self-hosted path is not that your organization stops using AI. It is that your staff use consumer tools on their own phones, with your data in them, because no approved path exists.

This course gives you one specific ability: to turn "compliance said no" into a list of technical controls, cost the architecture that satisfies them, and defend it to the two people who will interrogate it. Your auditor, who wants evidence. Your CFO, who wants the total number. Most leaders in your seat can do neither, which is why the project is still parked.

What you’ll learn

Go from a parked AI project to a costed, auditable architecture that runs entirely inside your perimeter, and that you can defend to anyone.

  • Map every objection: data egress, retention, training on your inputs, subprocessors, residency, missing BAAs, auditability.

  • Work the regimes that actually bind you: HIPAA, GDPR Article 28, ITAR, DORA, FedRAMP.

  • Leave session one with your own organization's objection-to-control mapping written down.

  • Model capability by size, stated honestly, including where the gap to frontier APIs still matters for your workload.

  • Quantization in plain language: why a model at a third the size loses almost nothing.

  • Run a capable model on your own laptop, disconnect from the network, and keep working.

  • Hardware tiers from a laptop to multi-GPU servers, and what each one actually buys you.

  • Ollama for development, vLLM for production, and why that difference matters in practice.

  • Run the break-even math against API pricing at your own real volume.

  • An air gap is isolated weights, disabled telemetry and controlled updates. Most deployments fail on the second one.

  • Model provenance, secrets and dependency management with no network to pull from.

  • Verify your own local stack is not quietly phoning home.

  • Why the tool call is the new trust boundary, and where your data really travels in a retrieval system.

  • Authorization on every call, per-operation ephemeral credentials, sandboxing, tamper-evident audit logs.

  • Design the approval gate and audit record for one of your own workflows.

  • What an auditor asks and what you must produce: log design, retention, model versioning, reproducibility.

  • Total cost of ownership including the operating costs most self-hosting proposals leave out.

  • Present a reference architecture and business case built for your own organization.

Learn directly from Thomas

Thomas Underhill

Thomas Underhill

Product & Eng Leader (AWS, VMware, HashiCorp, NGINX) · Adjunct Professor

Previously at
Amazon Web Services
HashiCorp
VMware
NGINX
University of California
See all products from Thomas

Who this course is for

  • VP or director of engineering whose AI pilot worked, then stopped the moment privacy or compliance asked where the data would go.

  • CISO or security architect who keeps having to block AI deployments and wants a defensible way to say yes instead.

  • Principal engineer or architect handed the question "can we run this ourselves?" and expected to answer it with real numbers.

Prerequisites

  • An Apple Silicon Mac with 16GB of memory, and rights to install software on it

    You run a real model locally in week one. Leave 50GB free. Labs and support are macOS only; on Linux or Windows you translate the commands.

  • One real system at your organization that is blocked, and the regime behind it

    Every exercise and the final deliverable are built for your environment, not a case study. You need to name the regime, not master it.

  • Comfort in a terminal. No programming or AI background needed.

    You paste commands, read output, and edit a config file. If that sounds ordinary, you are set. On the build track a model writes the code.

What's included

Thomas Underhill

Live sessions

Learn directly from Thomas Underhill in a real-time, interactive format.

Your Own Reference Architecture

You do not leave with notes. You leave with a costed reference architecture for your own environment: the objection-to-control mapping, the model and hardware tier, the deployment and audit practices, and a total cost of ownership case. If you took the hands-on track, it has evidence attached in an appendix rather than intentions.

Run It On Your Own Machine

In week one you install a capable model on your laptop, disconnect from the network, and keep working. No cloud spend, no GPU purchase, no vendor involved. Most people in regulated industries have never seen this work, and twenty minutes of watching it run changes what the rest of the course means.

1:1 Session with Thomas

One 30 minute session on your architecture, bookable from week three once you have a draft worth reviewing. Bring the part you would rather not raise in front of the group: your regulator, your data, the objection nobody has answered. I have built this stack and will tell you plainly whether your design holds.

Group Office Hours, Off the Record

Two optional sessions, deliberately not recorded, so the room can be honest about what is actually blocked and why. This is where a hospital CTO says the real problem was the auditor's third question, and a bank CISO says the same thing happened to them. That conversation does not happen on a recording.

Post-course access

Go back to course content and recordings whenever you need to.

Certificate of completion

Share your new skills with your employer or on LinkedIn.

Peers You Can Actually Ask

A closed channel with the people who did this work alongside you, still there after the course ends. Healthcare, finance, government and legal, the same problem under different regime names. Nowhere else can you ask how someone answered their auditor without disclosing a gap in your own program.

Two Ways to Do the Hands-On Work

Every week has an optional exercise and you choose how to take it. Build track: point a local coding assistant at the week's problem and construct it. Review track: I hand you working code with realistic flaws planted in it, and you find them. Switch week to week.

It Does Not End at Session Eight

Two things arrive after session eight. In December, a lesson on how to tell whether what you built actually works, with the exercise (model evals) that measures it. In January, a sixty minute alumni session, not recorded, where you come back and say what happened, including if the answer is nothing. Both free, and both are why people stay in touch.

Maven Guarantee

Your purchase is backed by the Maven Guarantee.

Course syllabus

11 live sessions • 14 lessons • 4 projects

Week 1

Oct 20—Oct 25

    Module 1: The constraint, and what is actually possible

    • Oct

      20

      Session 1: Why your compliance team said no

      Tue 10/203:00 PM—4:30 PM (UTC)
    • Oct

      20

      Session 2: What you can actually run

      Tue 10/203:00 PM—4:30 PM (UTC)
    • Oct

      23

      Optional: Group office hours (optional, not recorded)

      Fri 10/233:00 PM—3:45 PM (UTC)
      Optional
    3 more items

Week 2

Oct 26—Nov 1

    Module 2: The infrastructure

    • Oct

      27

      Session 3: Hardware, serving, and what it costs

      Tue 10/273:00 PM—4:30 PM (UTC)
    • Oct

      29

      Session 4: Air-gapped and disconnected deployment

      Thu 10/293:00 PM—4:30 PM (UTC)
    5 more items

Schedule

Live sessions

3 hrs / week

Live sessions will be recorded

    • Tue, Oct 20

      3:00 PM—4:30 PM (UTC)

    • Tue, Oct 20

      3:00 PM—4:30 PM (UTC)

    • Fri, Oct 23

      3:00 PM—3:45 PM (UTC)

Projects

1-3 hrs / week

Each week you will complete hands-on lab style activities that help you build towards a Capstone project

Async content

1-3 hrs / week

Asynchronous supplementary videos containing demos, walkthroughs, and explainer materials as well as brief reading materials with examples.

Frequently asked questions

Maven for Teams

Reimbursement

Get your company to pay

Everything L&D needs: email template, receipts, and certificate of completion.

Get reimbursed

Team discount

Learn with your teammates

Save 20%+ when 2 or more teammates enroll in the same cohort.

Save 20%+ with a team

Private cohort

Run a cohort for your org

A dedicated cohort with a custom schedule and curriculum, tailored to your team.

Book a private cohort

$1,900

USD

Oct 20Nov 23
Enroll