On-Device LLMs: Ship Models to the Edge in 2026

Hosted by Ehsan Gazar

Tue, Aug 4, 2026

5:30 PM UTC (30 minutes)

Virtual (Zoom)

Free to join

Invite your network

Go deeper with a course

From Senior to Staff: Master the Architecture Skills That Get You Promoted
Ehsan Gazar
View syllabus

What you'll learn

Know when local beats a frontier API

Score a task on latency, cost, privacy and offline needs to decide if it belongs on-device at all.

Quantize a model without wrecking quality

See what 4-bit and 8-bit quantization cost in accuracy, and where the quality cliff actually falls.

Ship an edge model to real hardware

Run a small model locally with llama.cpp or WebGPU and measure tokens per second on device.

Why this topic matters

Not every feature needs a frontier API. A quantized small model on-device or at the edge can win on latency, cost, privacy and offline use for a real class of tasks, yet most teams reach for the biggest model by reflex. This lesson shows where local wins, where the quality cliff falls, and how to ship one to real hardware. You leave with a right-sizing rule and a runnable local setup.

You'll learn from

Ehsan Gazar

Staff Software Engineer at Tipalti

I'm a Staff Software Engineer with 16 years building and scaling production systems across fintech, SaaS, and enterprise software. I've made hundreds of architectural decisions in systems that had to survive real traffic, incidents, and org politics.

I've run 500+ mentorship sessions with senior engineers at a 5.0 rating, helping them close the gap between writing code and thinking at the architectural level. I've also taught 10,000+ students and distilled what separates engineers who get promoted from those who stay stuck.

I'm not an academic. Every framework here comes from real decisions or mistakes I had to recover from. I’ll teach you to think like a Staff Software Engineer: not “what’s the right answer,” but “what are the trade-offs, and what survives contact with reality.”

See all products from Gaz

Sign up to join this lesson

By continuing, you agree to Maven's Terms and Privacy Policy.