Lesson 1: Why You Can't Say What Went Wrong

Learning goals:

• Understand why a single user-visible symptom can hide multiple indistinguishable root causes
• Explain how non-determinism breaks the traditional "reproduce it and set a breakpoint" debugging intuition
• Recognize the different shapes errors take in agent systems: cascading failures, trajectory divergence, cross-turn compounding, and multi-agent emergence

Prerequisites: Complete the first 10 courses in this series, be able to hand-write a harness loop driven by stop_reason, understand that evaluation tracks only give you pass/fail | Next: Lesson 2 >>

A Tuesday afternoon you can't explain

Your internal research agent has been live for two weeks. Tuesday afternoon, the ops team forwards you a user report:

I asked it to find our pricing plan from last year. It said it couldn't find anything. But that document is right there in the knowledge base—took me two seconds to pull it up myself.

You open the session. Two things on screen: the user's question and the agent's final response, "I didn't find any relevant materials." What happened in between? You have nothing.

So you start guessing.

Did it construct a bad search query—like taking the user's natural-language question and shoving the whole thing into retrieval verbatim instead of extracting keywords? Or maybe it found results but picked the wrong sources, reading the two least relevant hits out of eight and concluding "nothing here"? Or the retrieval tool threw an error, and the agent interpreted the failure as "no information in this direction" and moved on?

All three guesses look identical from the user's side: the agent can't find information that's obviously there.

This isn't unique to you. When Anthropic's team did a retrospective on their multi-agent research system, they wrote down the exact same problem: users would report agents "not finding obvious information," but they couldn't see why. Were the agents using bad search queries? Choosing poor sources? Hitting tool failures?¹ Same three questions, no answers.

The evaluation track only answers "did it break"

Your first instinct is probably to open the system you built in the previous course (Course 10 in this series): terminal-state scoring, validators, evaluation sets. That's the right first step. You add this real user question to the evaluation set, write a criterion "the answer must cite the pricing document," and run it.

Result: one red line. fail.

That red line is useful—it turns a subjective complaint into a reproducible, regression-testable verdict. But it doesn't answer the question you actually need answered right now: why. The scorer looks at terminal state. Terminal state is "didn't cite the document." Whether that "didn't cite" came from a bad query, source selection, or a swallowed tool error—the scorer doesn't care and has no way to care. It stands at the finish line holding up a scorecard.

This course fills in the middle section. Using a framework this course coined:

Verification tells you whether it broke. Observability tells you why.

This isn't a line from any official documentation—it's this course's framing to tie together the next five lessons, backed by two real experiences. One comes from that multi-agent system retrospective: adding full production tracing let them diagnose why agents failed and fix issues systematically¹. The other comes from the tooling engineering side: analyzing the raw transcripts your evaluation agent leaves behind can help you probe why agents do or don't call certain tools². Both sentences point to the same thing—only when the process leaves a trail can you ask "why."

Why "reproduce it and set a breakpoint" doesn't work here

First, definitions. A deterministic system means: give it the same input, it produces the same output every time. A non-deterministic system is the opposite—agents are this kind of system. In computing, deterministic systems produce the same output every time given identical inputs, while non-deterministic systems—like agents—can generate varied responses even with the same starting conditions².

This pulls the rug out from under traditional debugging. Agents make dynamic decisions and are non-deterministic between runs, even with identical prompts. This makes debugging harder¹. Traditional evaluations often assume that the AI follows the same steps each time: given input X, the system should follow path Y to produce output Z. But multi-agent systems don't work this way. Even with identical starting points, agents might take completely different valid paths to reach their goal¹.

Here's what it looks like in practice:

Same question, same prompt version, two runs
Run A                                  Run B1 search("2024 pricing plan")          1 search("pricing plan 2024 internal docs")2 read_doc(doc_17)                     2 search("pricing archive")3 respond (citing doc_17)              3 read_doc(doc_09)                                       4 read_doc(doc_17)                                       5 respond (citing doc_17 and doc_09)

Neither path is wrong, and both terminal states would pass. But if the failure happened in Run A step 2, and you re-run ten times and get nine Run Bs, those nine runs don't help.

Breakpoints are equally useless. Breakpoints sit on code lines, with the precondition that "the second execution will hit this line with the same context." But where agents go wrong is often not in your code—it's in a model decision. And even if you wanted to pause, you wouldn't know which turn to pause at: this time it breaks on turn 3, next time maybe turn 11, or maybe it doesn't break at all.

You might think of turning down sampling temperature or recording and replaying tool responses. These engineering techniques do reduce some jitter, and this course doesn't discourage you from using them. But they change the run you're doing in your lab, not the one that already failed in production—that one is gone, and the only thing it left behind is its records.

Anthropic's team took a different approach. They call it "think like your agents": build a simulation using the exact same prompts and tools from the system, then watch agents work step-by-step. This immediately revealed failure modes: agents continuing when they already had sufficient results, using overly verbose search queries, or selecting incorrect tools¹.

The key isn't "reproduce the same run"—it's "see each step." That's where observability diverges from traditional debugging.

The errors agents throw come in different shapes

Even if you accept "you need records," there's another layer to understand: the shape of errors in agent systems isn't the same as traditional bugs.

In traditional software, a bug might break a feature, degrade performance, or cause outages. In agentic systems, minor changes cascade into large behavioral changes, which makes it remarkably difficult to write code for complex agents that must maintain state in a long-running process¹.

Within a single run, the most typical shape is trajectory divergence. Agents are stateful and errors compound. The compound nature of errors in agentic systems means that minor issues for traditional software can derail agents entirely. One step failing can cause agents to explore entirely different trajectories, leading to unpredictable outcomes¹.

Turn 3: search times out, tool returns "Error: upstream timeout"        ↓Agent reads it as "no information in this direction"        ↓Switches to a completely different retrieval angle        ↓The next 12 turns all grow on this wrong branch        ↓Terminal state: a seemingly complete report, all sources from tangential material

Notice how small the first link in that chain is: one timeout. In a traditional service it might just be a retry log entry. Here it rewrites the next twelve turns, and the terminal-state report doesn't error out, doesn't crash, reads perfectly fluently.

The second shape is cross-turn compounding. Agents are stateful and errors compound. Agents can run for long periods of time, maintaining state across many tool calls. This means we need to durably execute code and handle errors along the way¹. That's also why you can't just restart from the beginning when errors occur: restarts are expensive and frustrating for users. Instead, we built systems that can resume from where the agent was when the errors occurred¹. The implication for you is direct: if the state at failure time wasn't recorded, the question "where to resume from" has no answer.

The third shape only shows up in multi-agent systems: emergent behavior. Multi-agent systems have emergent behaviors, which arise without specific programming. For instance, small changes to the lead agent can unpredictably change how subagents behave. Success requires understanding interaction patterns, not just individual agent behavior¹.

What these shapes look like when they all converge, that team's early version gave a very direct answer: early agents made errors like spawning 50 subagents for simple queries, scouring the web endlessly for nonexistent sources, and distracting each other with excessive updates¹.

These three error types have something in common: viewed from the outside, the terminal state might just look like "answer quality is meh." You can't tell which one it is.

Abstraction layers hide the evidence

There's another layer of trouble, and it doesn't come from the model—it comes from your tooling.

Agent frameworks make it easy to get started by simplifying standard low-level tasks like calling LLMs, defining and parsing tools, and chaining calls together. However, they often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug³. That's why the advice to developers is: start by using LLM APIs directly—many patterns can be implemented in a few lines of code. If you do use a framework, ensure you understand the underlying code. Incorrect assumptions about what's under the hood are a common source of customer error³. (That article includes an editorial note saying its tool ecosystem descriptions are outdated, so we're citing the principles here, not treating it as current selection guidance.)

For you, this is actually good news. You hand-wrote the harness loop in Course 7 of this series. You don't have that layer of abstraction:

Evidence flows through this loop every time, without fail. The problem is after it flows through, it's gone. You have no obstruction, but you also have no retention—from the perspective of asking for evidence after the fact, those two are equally painful.

So this course's diagnosis is: half the difficulty of debugging agents comes from the objective fact of non-determinism, and the other half comes from "things that could have been recorded weren't." The first half can't be changed. The second half is in your hands.

The way forward: what the next five lessons give you

That multi-agent system's team put this on the same tier as prompt engineering and tool design: getting this right relies on careful prompting and tool design, solid heuristics, observability, and tight feedback loops¹. The retrospective has an even more direct line—they focused on a fast iteration loop with observability and test cases¹.

Notice the "test cases" half: observability isn't here to replace the evaluation track. They're two ends of one carrying pole. Evaluation tells you whether this run broke and whether the change made things better. Observation tells you why this run broke and which part to change.

The next five lessons proceed in this order:

• Lesson 2 lays the foundation: raw records are the first-hand evidence. What the agent says about itself doesn't count—what it omits is often more important than what it includes.
• Lesson 3 turns each step into data: structured logs and metrics. The same metrics Course 10 used for scoring, this course uses for diagnosis.
• Lesson 4 threads scattered records into a tree: read all the model requests and tool executions triggered by one prompt as a single unit, with subagent calls nested inside the parent.
• Lesson 5 installs probes at the lifecycle checkpoints of the loop, walks through the locate-under-non-determinism workflow: finding the first divergence from a pile of records.
• Lesson 6 hands-on: install a full observability layer on the harness you wrote in Course 7, tracing from a symptom you "can't explain" to the specific step that diverged.

This lesson only names them without expanding: how to read records is Lesson 2's job, how to design metrics is Lesson 3, what structure a trace has is Lesson 4.

When you don't need the full stack

Not every agent needs all of this. A one-off script—run once, glance at the output, delete—giving it logs, metrics, and traces is pure waste.

The judgment criterion is simple: observability investment should be proportional to "how long it would take you to explain why after something breaks." If you run it yourself, watch it yourself, and it costs nothing to re-run when it fails, don't record. If someone else uses it, if it runs long, if you'd spend half a day digging through chat transcripts when it fails, record from day one.

There's one line that has nothing to do with scale: if the agent can take action—write files, send requests, spend money—don't skip testing. The autonomous nature of agents means higher costs, and the potential for compounding errors. We recommend extensive testing in sandboxed environments, along with the appropriate guardrails³.

As for knowing you got it right: the key to success, as with any LLM features, is measuring performance and iterating on implementations³. And the precondition for measurement is having something to measure—which loops back to what this course is solving.

💻 Exercises

Recap

• A single user-visible symptom often hides multiple causes that are completely indistinguishable from the outside. When users report agents "not finding obvious information," you can't tell whether it's bad search queries, poor source selection, or hitting tool failures¹.
• The evaluation track answers "did it break." This course uses a framing it coined to clarify the division of labor: verification tells you whether it broke, observability tells you why. Backing it are two real experiences—adding full production tracing let them diagnose failures and fix systematically¹, and reading raw transcripts can help you probe why agents do or don't call certain tools².
• "Reproduce and set a breakpoint" fails because agents make dynamic decisions and are non-deterministic between runs, even with identical prompts¹. Deterministic systems give the same output for the same input; agents, as non-deterministic systems, don't guarantee this². Traditional evaluation's assumption—"input X follows path Y produces output Z"—doesn't hold either. Identical starting points can produce completely different valid paths¹.
• Agent error shapes differ from traditional bugs: in traditional software bugs break a feature, but in agentic systems minor changes cascade into large behavioral changes, making it remarkably difficult to write code for complex agents that must maintain state¹. One step failing can cause agents to explore entirely different trajectories, leading to unpredictable outcomes¹. Agents are stateful and errors compound, so you need to durably execute and handle errors along the way¹. Multi-agent systems also have emergent behaviors—small changes to the lead agent can unpredictably change subagent behavior; what you need to understand is interaction patterns, not just individuals¹. The early version once spawned 50 subagents for simple queries, scoured the web endlessly for nonexistent sources, and distracted each other with excessive updates¹.
• Abstraction layers hide evidence: frameworks often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug³. The advice is to start with LLM APIs directly, and if using a framework, understand the code underneath³. Your hand-written harness has no obstruction, but also no retention.
• The way forward is to treat observability and tight feedback loops as first-tier requirements¹, built into a fast iteration loop with observability and test cases¹. The next five lessons in sequence: raw records, logs and metrics, traces, hooks and locate workflow, hands-on installation.
• Judgment on scope: one-off scripts don't need the full stack. But if the agent can take action, do extensive testing in sandboxed environments with appropriate guardrails³. Knowing you got it right relies on measuring performance and iterating on implementations³.

Lesson 2 >>