Latency budgets, desktop permissions, SIP rules, plus the AI minefield—hallucinations, accent-biased transcriptions, token-burst rate limits, and model drift—turn “just add voice” into months of painful debugging even for senior engineers. WebRTC jitter ruins timing, loose prompts spark off-brand replies, and a single bad transcript cascades into wrong actions and angry users.

Over four weeks you’ll build three voice products—desktop, browser, and telephony—side-by-side with the maintainers of Vapi, Pipecat, LiveKit, and Whisper, plus Ivan and Nicolay, who’ve shipped these systems in production. You’ll master the shared scaffolding (stream segmentation, prompt stitching, cost/latency meters) and the AI guardrails (real-time validation, confidence scoring, speculative decoding) that keep voice assistants responsive, factual, and customer-safe across every channel and model.

But don't just trust us that Voice AI is here to stay and having it's "ChatGPT" moment.

"Humans interact with businesses in many ways, but one way hasn't changed much in almost 100 years—and that's phone calls.Today, over a trillion calls exist between a business and a customer....new voice models and conversational LLMs are now incredibly good ... startups are ... making voice AI bots that are indistinguishable from humans." - Gustaf Alströmer, YC -- in a call for Voice AI startups

"For enterprises, AI directly replaces human labor with technology. It’s cheaper, faster, more reliable — and often outperforms humans. Voice agents also allow businesses to be available to their customers 24/7 to answer questions, schedule appointments, or complete purchases...For consumers, we believe voice will be the first — and perhaps the primary — way people interact with AI." - Olivia Moore, a16z -- AI voice in consumer

There are already meeting bots that talk to you over Zoom, language coaches that help you learn Spanish in a webapp, ambient assistants that sit on your laptop, listen, and help you out when you need some input. "Voice UX" will keep growing in surface: We will move into cars, AR glasses, smart speakers, and other areas we haven't even considered yet.

Each channel breaks in its own way—latency on the web, permissions on desktop, SIP rules on telephony. One demo can’t teach you all of that.

That's why we build three separate voice AI applications in this course. A webapp, a native (MacOS) app, and a telephony app.

1. Native macOS Meeting Assistant – records your mic locally, takes live notes, pushes tasks to Notion, and pings you in Slack before deadlines. Learn how the most successful app to date (Granola) does it. Manual note-taking in back-to-back calls burns 6 h/week and important information still slips.

2. Web-based AI Sales Coach – simulates tough customers, scores every response, and shows real-time coaching tips without breaking flow. Learn how to live update UX based on an ongoing conversation. New reps take 6 months to hit quota; live coaching is expensive.

3. Telephony Booking Bot – calls clients, confirms appointments, handles DTMF/silence, and writes results straight into your CRM. Learn how to reliably call and handle diverse accents. Staff spend hours calling clients; no-show rate ~45 %.

Why you care as a student

- These metrics resonate with CTOs, PMs, and investors—your demo isn’t a toy.

- Each channel teaches a different “gotcha”: OS sandbox, browser jitter, telephony regs. Master once, reuse forever.

- Portfolio proof: three repos that shout “I can ship voice products anywhere users speak.”

After that, the next interface is just more plumbing.

The tools you’ll learn

Vapi – voice routing without IVR hell

Pipecat – low-latency audio transforms

LiveKit – WebRTC that survives bad networks

Whisper/Elevanlabs/AssemblyAI – fast, accurate transcription

OpenAI & Gemini Realtime – millisecond-level reasoning

Hands-on workshops with the engineers who wrote these libraries

Exclusive Access: Connect directly with the engineers building these tools through dedicated workshops and Q&A sessions. Learn from those who know these technologies best.

- LiveKit founder workshop: learn about WebRTC and how to make networking a breeze.

- OpenAI real-time API creators: learn how to best prompt real-time model.

- more workshops from speakers from ElevenLabs, AssemblyAI, Vapi, Pipecat will be added soon.

Prerequisites (read this)

- Comfortable in TypeScript/JavaScript (async/await, streams, React or similar)

- Basic REST & WebSocket chops

- Familiarity with Git and command-line tooling

If you’ve never shipped production code, this bootcamp will overwhelm you.