Conversational observability: evaluating and improving your GenAI chatbot

Learn how to set up a Claude Code project in Claude Desktop

Keep track of your work with session backlogs and handover skills

Connect Claude Code to Langfuse

Learn a new way to evaluate GenAI-chatbots, building on what you already know as a conversation designer

Discover evaluation as an ongoing cycle, so your process keeps working as the bot and its answers evolve.

Master a step-by-step method (collect, analyse, measure, experiment, improve) that makes your evaluation repeatable and easy to explain.

Set up logging to capture the conversational data that you need for insights.

Trace analysis and labeling: learn what to track in a generative chatbot

Apply the transcript-reading and pattern-spotting skills you already use in flow-based design.

Learn failure modes and metrics that are unique to generative bots, like hallucinations, over-confidence, and off-brand tone of voice.

Turn the patterns you spot in Analyse into repeatable, automated checks you can run anytime.

Create experiments to compare prompt variants against the same test conversations, so you know what works before it goes live.

Turn your outcomes into concrete prompt or retrieval changes, backed by the numbers.

Turn overwhelming Langfuse data into simple, custom views that product owners and content people can actually use.

Use Claude Code to build a custom dashboard on top of Langfuse and your chatbot, no analyst skills required.

Give your whole team a shared, at a glance view of how the chatbot is performing, so insights reach beyond the data team.