AI Analyst Lab

Cohort 2

Cohort 1

Cohort 3

Bootcamp 2

Bootcamp 4

Bootcamp 1

Bootcamp 3

Bootcamp 5

36 free lessons on evaluating, measuring, and improving AI-powered products. Full slide decks with written explanations. Weekly quizzes and a certification exam. What you'll learn<ul><li>Design observable AI systems with proper logging and trace architecture</li><li>Run structured failure analysis on production data</li><li>Translate AI quality metrics to user behavior and business results</li><li>Build evaluation workflows from offline tests through production monitoring</li><li>Make ship/iterate/pause/rollback decisions based on evaluation data</li></ul>

Free 6-week AI Evals Certification Course

Product managers who drive strategy, development, and launch of products.

Engineers and engineering leaders who build, deploy, and scale software systems.

Data scientists who apply statistical and ML techniques to extract insights from data.

Applying AI tools and techniques to professional workflows — assign alongside more specific AI topics.

Building autonomous AI agent systems that take actions, use tools, and orchestrate multi-step workflows.

Using data and metrics to measure performance, run experiments, and inform decisions.

Using AI coding assistants and tools to write, review, and ship code faster.

Transform from consumer of analytics to independent operator - asking sharp questions, validating answers, shipping decisions.

Principal Data Scientist @ Ontra 

Shane Butler

Principal Data Scientist @ Ontra | Ex-Stripe, Nextdoor, PwC, Appfolio

Data Science Leader |Ex-Microsoft

Sravya Madipalli

Data Science Leader@Superhuman (Prev. Grammarly)| Ex-Microsoft, eBay, Nextdoor

Head of Data @ Ontra AI | Ex-LinkedIn

Hai Guan

Head of Data @ Ontra | Ex-Nextdoor, LinkedIn, Pinterest, Meta

AI Analytics for Builders

Storyteller

Data Scientist

Lead Data Engineer

Data and Analytics Manager

Fractional Product Generalist

Sr. Data Science Manager

Co-Founder & CEO

Independent Consultant

Using Claude Code to build and ship software through an AI-powered command-line coding workflow.

Leave with a working agentic analytics system on your machine, a polished analysis, and the patterns and frameworks to make it all yours.

Claude Code Analytics Bootcamp

Director, Data Science 

Product Analyst

This is a hands-on weekend bootcamp (4 hours Saturday, 4 hours Sunday) where you build AI analysts powered by Claude Code. Not a lecture series. Not a demo you watch. You set up your own environment, build custom skills and agents, connect to real data, and leave with a working system that turns business questions into charts, narratives, and presentations in minutes.No Python. No SQL. No coding of any kind. Claude Code writes all the code. You direct the analysis.By Sunday evening, you will have a complete AI analyst repo on your computer: 45 skills, 27 agents, data connections, and a finished analysis you can share with your manager Monday morning. Everything runs locally. No vendor lock-in. No proprietary platform. Just a repo you own.AI is changing how teams make decisions with data. The bottleneck is no longer the analysis — it is knowing how to set up the tools. This bootcamp gives you that setup in a weekend. You will not learn about agentic systems. You will build one.

Build agentic data analysts for your company in one weekend. No code.

Set up Claude Code as your personal AI Analyst environment

Build custom skills that encode your analytical thinking

Build and chain agents into a multi-agent analytics system

Read and integrate your work into a production AI Analyst

Expand the system, then connect it to your team's tools

Product managers who are tired of waiting in the data team's queue. Stop waiting. Analyze it yourself.

Data scientists who want to multiply their output. You know how to do the analysis manually. Now automate it.

Engineers who want to make data-driven decisions without hiring a data team.

Stripe

Nextdoor

PwC India

Ontra

AppFolio

Microsoft

eBay

Grammarly

Superhuman

LinkedIn

Pinterest

Ask Better Questions, Make Faster Decisions - with AI as Your Analyst

Frame Problems Like a Senior Data Scientist

Design Metrics That Actually Drive Decisions

Diagnose Root Causes Without Guessing

Design Experiments That Ship

Turn Analysis Into Action With AI as Your Copilot

Synthesize Insights and Prioritize for Impact

Product Managers, engineers, and operators who need analytical independence and frameworks to measure success and drive decisions.

Designers and researchers stepping into strategy roles who need to quantify impact, run experiments, and influence with data-driven insights

Data professionals who want to frame better questions, delegate technical execution to AI, and drive impact with agentic analytics tools.

You'll learn to build a Claude Code and Codex AI analyst (45+ Skills and 25+ Agents) system you can trust, scale, and take to work. 

PwC España

You learned to use Claude Code for analysis. Now build the system behind it. This hands-on weekend bootcamp (4 hours Saturday, 4 hours Sunday) takes you under the hood of a production AI Analyst system with 45+ skills, 25+ agents, and multi-agent pipelines, and teaches you to build your own version from scratch.The problem every Claude Code user hits: how do you trust the output? A polished analysis can be completely wrong. Inflated numbers from a bad join, causal claims on observational data, a metric definition that changed two months ago. You won't know until it's too late. This bootcamp solves that. You'll build a multi-agent pipeline where a fresh-context Validator independently reviews every analysis. You'll add multi-layered context with freshness tracking so your AI never works with stale data. You'll wire Karpathy's auto-research loop so your system generates and validates its own context. And you'll add rules and hooks that enforce analytical rigor at the system level.Everything connects. Each module's output feeds the next. By Sunday evening, you take home two working repos (Claude Code + Codex), a complete AI analyst system you built yourself, and 10 reference guides.

Build production AI analyst system with automated context, multi-agent pipeline

How AI Analyst was built, under the hood

Context management + self-improvement loops

Validation, the #1 request from Bootcamp 1

Advanced MCPs + CLI automation

Best practices + practitioner workflows

Two repos + open source

Product managers already using Claude Code. You've built skills and agents. Now build a system that validates its own output.

Data scientists who know how to work with Claude Code. Now automate the full workflow: context loading, analysis, validation, and narrative.

Analysts and engineers comfortable with skills and agents who want to go deeper in Claude Code.

Learn practical, hands-on methods to evaluate, measure, and improve AI products using data science to make better product decisions.

Principal Data Scientist, AI Evaluations

Principal Data Scientist, AI Evaluations at Ontra

With the recent leaps in model capabilities, it is becoming clear that much of the AI evaluation work previously requiring humans to build and run can be automated through agentic systems. This course teaches you how to build those systems, and where human judgment and input is still required.The foundational work of designing eval frameworks, defining quality signals, and structuring failure analysis remains critical. But the execution of that work, running evaluations, scoring results, iterating on improvements, and surfacing what matters, can now be handled by agentic workflows that operate continuously and at scale.This course equips you with the capability to build, test, and deploy automated evaluation systems using Claude Code. You will leave with working systems, not just conceptual understanding.The focus is on automation that is trustworthy. Systems that score their own output, flag uncertainty, and know when to surface decisions to a human rather than proceeding autonomously.The foundational content on AI evals for product development is available free at <a target="_blank" rel="noopener noreferrer nofollow" href="https://aianalystlab.ai">aianalystlab.ai</a>. If you are new to AI evals, start there.

Build the agentic systems that automate your AI evaluation workflow from end to 

Build agentic evaluation workflows in Claude Code

Automate scoring and quality assessment

Implement iterative improvement loops

Scale experiment execution

Integrate with production data and decision-making

You will get the most out of it if you are actively working on, or about to ship, AI features in a real product.

Real-World AI Product Experience

Product Managers: Want automated eval systems for your team? Learn what is possible, reliable, and where human oversight stays essential.<pre><code class="language-markdown"></code></pre>

Data Scientists &amp; Analysts: Spending more time running evals than learning? Build systems that handle execution so you focus on decisions.

Engineers and ML Engineers: val bottlenecks slowing your release cycle? Automate repetitive work and ship with speed and confidence.

Vibe Code Annotation UIs for AI Analytics Evals

Validate Claude Code Analytics Output

Develop North Star Metrics with AI

Publish Analysis Everywhere with Claude Code

Agentic Experiment Design & Analysis

Cut Wasted Analysis using AI

Set Up Claude Code for Analytics for the First Time

Principal Data Scientist @ Ontra

Principal Data Scientist @ Ontra | Ex-Stripe, Nextdoor, PwC

Data Science Leader | Ex-Microsoft

The course fee for this workshop is $50. Use PROMO code: LEARN50. Please read below to make sure this workshop is a good fit for you.***Who this workshop is NOT for- You've already installed Claude Code and are comfortable working with it in terminal.- You've already worked with the AI Analyst system and are familiar with how it runs.- You've built your own skills and agents in Claude Code terminal before.If any of those describe you, this workshop will feel too basic.***You've seen what Claude Code can do. But getting it running on your machine, pointed at real data, producing a real chart? Well that gap between "cool demo" and "it's running for me" can feel enormous.This workshop closes that gap in one hands-on morning.You walk in with zero Claude Code experience. You install Claude Code on your machine, clone the AI Analyst repo, point it at a dataset, and ship a real analysis.This is also where you start building community. A room of people who keep building together after Saturday, share what's working, learn from each other.No prior experience required! No coding required!Just a willingness to learn, try something new, and meet some new people :)

Product managers tired of waiting in the data team's queue. Stop waiting. Build an AI analyst that does the analysis for you.

Data Analyst and Scientists who want to automate repetitive analysis. You know what good analysis looks like. Now build agents that execute.

Engineers who want data-driven decisions without a data team. Build the system yourself in one morning. No statistics degree required.

Learn directly from AI Analyst Lab

AI Analyst Lab

Courses

Claude Code Analytics Bootcamp

AI Analytics for Builders

Advanced AI Analytics Bootcamp

Automate AI Evals with Claude Code

Workshops

Intro to Claude Code Analytics

Lightning Lessons