Cohort 1

CEO & Co-Founder

Jack Shaw

Product managers who drive strategy, development, and launch of products.

Engineers and engineering leaders who build, deploy, and scale software systems.

Measuring and evaluating AI model performance, quality, and reliability for production use cases.

Startup founders building new businesses, from MVP to scale.

Applying AI tools and techniques to professional workflows — assign alongside more specific AI topics.

Building autonomous AI agent systems that take actions, use tools, and orchestrate multi-step workflows.

Using AI coding assistants and tools to write, review, and ship code faster.

Designing and optimizing the context window for LLM applications — prompt structure, memory, and information retrieval.

Operating and deploying large language models in production — fine-tuning, inference, monitoring, and cost optimization.

Build an agentic harness end-to-end, hands on, and walk away able to brief the goal, steer the run, and verify on your own repo confidently.

Founder, SpecStory • Ex-CPO, Pluralsight

Pluralsight

GitHub

Dropbox

Google

AlixPartners

Greg Ceccarelli

Co-Founder, SpecStory • Ex-CPO Pluralsight • Data at GitHub, Dropbox & Google

Founder, SpecStory • Ex-CTO, Hexawise

Hexawise

Carrot

The University of North Carolina at Chapel Hill

Sean Johnson

Co-founder, SpecStory • CTO & Co-founder, Hexawise • Product, Testing, AI

Founder, SpecStory • Ex-CPO, Docker

Docker

DigitalOcean

Yesware

Jake Levirne

Co-Founder, SpecStory • Ex-CPO Docker  • Product at Digital Ocean, YesWare & IBM

Hardcore Agentic Engineering for builders who ship

Co-Founder • Ex-CPO Pluralsight • Data at GitHub, Dropbox & Google

<h3>Why I enrolled</h3>
<p>I keep running into the same problem whenever I ask LLMs to judge LLM output. Deterministic rules are easy enough to check, but many products I care about depend on judgments that do not reduce cleanly to a linter. We may have strong examples and ground truth, yet the model's evaluation still drifts. I've seen this in clinical trial evaluation and now in work around frame ownership. Who actually supplied the reasoning and judgment in work produced with AI?</p>
<p>My goal for the course is to get better at designing reliable checks for that middle territory and find where my current approach breaks down. For Prove It, I'm using GAiDFLY, a voice agent that conducts a Socratic defense of a student's paper. I'm building an offline gate that looks at where the reasoning in that defense came from. A session can end with a strong student answer even when GAiDFLY introduced the underlying reasoning. I want the gate to catch that, cite the turns that matter, and leave ambiguous cases for human review. It will not prove what a student understands internally. But it should give the reviewer better evidence about whether the agent crossed from questioning into doing the student's thinking.</p>
<p></p>

    <h3>A breakthrough moment</h3><p class="p2-tall">My main takeaway is that process evidence and software-result evidence answer different questions. The Project 1 gate proved that the candidate passed its checks. It did not prove that the worker stayed inside the supplied brief. Project 2 helped me separate those claims and attack a weak check with a deliberate wrong case. I still want to learn how to check work that needs human judgment. For example, I want reliable evidence that shows whether an agent supported a student’s reasoning or supplied the reasoning itself.</p>

<p></p>
<h3>What I’ll apply</h3>
<ul class="list-disc ml-8"><li><p class="p2-tall">Define “done” before every meaningful agent run, including the exact checks and evidence I will accept.</p></li><li><p class="p2-tall">Move verification outside the agent’s control so the agent doing the work cannot alter the checks or declare itself done.</p></li><li><p class="p2-tall">Attack my own checks with realistic wrong results and make sure those failures turn the evidence red.</p></li><li><p class="p2-tall">Turn failed runs into durable improvements (stronger checks, clearer repository instructions, or reusable evaluation cases) and keep those changes only when later runs actually improve.</p></li></ul>
<p></p>

Certificate

Learn more about this course