Lesson 1: 'Looks Done' Isn't 'Is Done'

Learning goals:

• Explain why Claude stops when the work looks done, and who ends up doing the verification when that happens
• Use the contrast between deterministic and non-deterministic systems to explain why traditional testing's "input X, path Y, output Z" assumptions break down for agents
• Given a completion report, distinguish which statements are assertions vs. evidence, and identify what's still missing for sign-off

Prerequisites: Complete the first 9 courses in this series; write stop_reason-driven harness loops by hand; understand checkpoints and recovery | Next: Lesson 2 >>

Shipped on Tuesday, Broken on Friday

Tuesday afternoon, you ask an agent to add a "bulk import users" feature to an internal admin panel: upload CSV, parse it, validate fields, write to database. You spell out the requirements clearly, then head into a meeting.

When you get back, the session has stopped. The last message looks like this:

Done.
- Added src/importer/parseCsv.js for parsing and field mapping- Added src/importer/validate.js to check email format and required fields- Wired up batch insert at POST /api/users/import
Implementation follows existing module code style. No errors during execution.

You skim the diff. Functions are cleanly separated, naming matches neighboring modules, edge cases look considered—empty files return an explicit error, the email regex isn't obviously broken. You merge it. You ship it.

Friday afternoon, operations posts in the channel: "Why did we just import 400 empty users?"

The cause is straightforward. Operations generated that CSV by doing "Save As" from Excel, which added a BOM—three invisible bytes  that Excel likes to prepend to UTF-8 files. So the first column name got parsed as email instead of email, the entire field mapping fell through, and every row became "all fields are undefined." What about that validation layer? It checked "is the email format valid," but undefined took a different branch and was treated as "this column wasn't filled in," so it passed.

Nobody cut corners here. The agent wrote code that runs. It tested itself with a CSV it generated—and of course its own CSV doesn't have a BOM. When you reviewed the diff, you were checking "is this code written correctly," not "what happens when this code meets real-world input." Both sides tried their best. The gap still happened.

The problem is in the moment it stopped. When the agent stopped, what it had was "I wrote it, I read it once, looks fine." It didn't stop at "I've confirmed it's done." It stopped at "it looks done." And from the conversation history, you can't tell the difference.

It Stops Where Things Look Done

The Claude Code docs spell it out plainly: Claude stops when the work looks done; without a check it can run, "looks done" is the only signal available, and you become the verification loop: every mistake waits for you to notice it¹.

This sentence is worth reading twice, word by word. It's not saying "Claude sometimes cuts corners," or "the model isn't capable yet." It's describing a structural fact: if nothing in the entire pipeline can produce an objective result, then "looks done" is the only signal that exists in this system. The model can only make decisions with that signal. It has nothing else.

The same docs give this phenomenon a name: the trust-then-verify gap—Claude produces a plausible-looking implementation that doesn't handle edge cases¹. In plain English: you trust first (code looks good), and verification either doesn't happen or happens too late (Friday afternoon, in the ops channel). The BOM example above is the standard form of this gap: not wrong code, but nobody asked "what happens with an Excel-exported file?"

There's a second layer here that's easy to miss. The docs' suggested fix ends with: if you can't verify it, don't ship it¹. The emphasis isn't on "verify"—it's on "don't ship." It acknowledges that some things you just can't verify. When you can't, the correct move isn't "trust your gut this one time." It's narrow the scope, change the requirement, or hold off on shipping.

Assertions vs. Evidence: What's the Difference?

Go back to that completion message. Break it into individual sentences, and ask the same question for each one: can I confirm this sentence without reading code, using only what it's shown me?

• "Added src/importer/parseCsv.js"—you can confirm it. Whether the file exists is checkable at a glance. This is evidence (though the weakest kind).
• "Implementation follows existing module code style"—you can't confirm it. This is the model's aesthetic judgment. Assertion.
• "No errors during execution"—sounds like evidence, but it's actually an assertion. It's saying the tools it called didn't throw exceptions, not that the output is correct. All tools returning success while the result is completely wrong—totally possible.
• "Check email format and required fields"—you can't confirm it. This describes intent, not behavior. What that regex actually permits or rejects? This sentence says nothing about it.

Where's the line? Evidence is something a second person can re-run exactly the same way: a command plus its raw output, an exit code, a list of failed test names, a screenshot, a before/after numeric comparison. Assertions are things you can only choose to believe or not: "logic is correct," "should be fine," "already optimized," "won't happen again."

The official docs draw exactly this line: have Claude show evidence rather than asserting success—the test output, the command it ran and what it returned, or a screenshot of the result. Reviewing evidence is faster than re-running the verification yourself, and it works for sessions you weren't watching¹.

That last half-sentence is the key. If you were watching the whole time, the "assertion vs. evidence" distinction doesn't buy you much—you saw it yourself. But the moment you've looked away, all that's left in the conversation history is text, and in text, assertions look just as confident as evidence.

Why Agents Especially Hit This Problem

We see "looks right but is wrong" in traditional software too. Why does it warrant a dedicated lesson for agents?

Because traditional testing rests on an assumption that agents don't satisfy.

Start with definitions. In computing, deterministic systems produce the same output every time given identical inputs, while non-deterministic systems—like agents—can generate varied responses even with the same starting conditions². This isn't "has bugs so it's unstable." This is how it works. Even if you change nothing in your prompt, the decisions across two runs aren't guaranteed to match³.

So the premise of traditional evaluation collapses. Traditional evaluations often assume that the AI follows the same steps each time: given input X, the system should follow path Y to produce output Z³. Multi-agent systems don't work this way. Even with identical starting points, agents might take completely different valid paths to reach their goal—one agent might search three sources while another searches ten, or they might use different tools to find the same answer³.

Here's what that looks like concretely:

Same task, same prompt, two runs
Run 1: read_file(schema.sql) → grep("user_id") → edit(models/user.js)        → run_tests → done
Run 2: list_dir(src/) → read_file(models/user.js) → read_file(models/order.js)        → edit(models/user.js) → edit(models/order.js) → run_tests        → run_tests → done

You can't call either trajectory "wrong." The second run read an extra file, modified an extra place, and ran tests twice—maybe it took a detour, or maybe it caught coupling that the first run missed. If you write an assertion that says "must read schema.sql first," the second run fails—but the second run might've done better work.

Checking the trajectory against a prescribed script doesn't work here: because we don't always know what the right steps are, we usually can't just check if agents followed the "correct" steps we prescribed in advance³.

Add one more layer: errors in agent systems compound. A minor bug in traditional software, when it hits an agent, can derail the entire task—one step failing can cause agents to explore entirely different trajectories, leading to unpredictable outcomes³. This isn't like a traditional program where "one function returns a bad value, it propagates up." An agent takes a bad result and makes new decisions based on that bad result: misread a file, it might conclude "this module doesn't exist" and create a new one; then it keeps working around that new module. By the time you see the final output, the error isn't in its original spot anymore. It's grown into something else.

Anthropic's own conclusion lands here: the autonomous nature of agents means higher costs, and the potential for compounding errors. We recommend extensive testing in sandboxed environments, along with the appropriate guardrails⁴. And one more direct line—the LLM will potentially operate for many turns, and you must have some level of trust in its decision-making⁴.

Notice the phrasing "some level of trust." It's not saying "you have to trust it." It's saying this trust has to come from somewhere. And trust has only two sources: you watched it yourself (so the agent didn't save you any time), or something watched it for you. This entire course is about the second kind.

The Way Out: Give It a Check It Can Run

All that setup lands on one sentence: give Claude a check it can run—tests, a build, a screenshot to compare. It's the difference between a session you watch and one you walk away from¹.

How does the difference arise? Give Claude something that produces a pass or fail, and the loop closes on its own. Claude does the work, runs the check, reads the result, and iterates until the check passes¹.

You can map this sentence back to the harness loop from course 7 in this series. First look at where your current loop stops:

What does end_turn mean? It means the model thinks it's done talking for this turn. That's all. It doesn't mean the work is correct, and it doesn't even guarantee the response is complete—this loop only recognizes tool_use; if stop_reason becomes anything else it'll exit, including when output got cut mid-sentence by max_tokens. Nothing in the exit condition relates to "quality of output."

So what does wiring in a check look like? Two positions work.

Position one: make the check into a tool it can call, let it run inside the loop:

Position two: add a gate after the loop exits; don't trust its self-report, run it yourself:

The code itself has no tricks. The key is the exit condition changed owners: from "the model says it doesn't want to call more tools" to "a piece of deterministic code returned 0." The former is the model's self-assessment. The latter isn't.

So what can "check" be? The official docs give a broader range than you might expect: the check is anything that returns a signal Claude can read in the conversation—a test suite, a build exit code, a linter, a script that diffs output against a fixture, or a browser screenshot compared against a design¹.

Explain "fixture": it's a "golden answer file" you saved ahead of time; after running, you compare output to it, and it can't differ by even one character. Sounds blunt, but for "output format must be stable" kinds of tasks, it's the simplest and most reliable form of check.

This line of thinking aligns with Anthropic's recommendation for agent execution: during execution, it's crucial for the agents to gain "ground truth" from the environment at each step (such as tool call results or code execution) to assess its progress⁴. Notice "from the environment"—not from its own reasoning. The model's reasoning is self-generated. The environment's return values aren't.

The Next Five Lessons Solve What

With "give it a runnable check" as the main thread, the remaining questions become concrete.

Lesson 2: What to verify. Since checking the trajectory against a prescribed script doesn't work, what do you check? Answer: end state first—evaluate whether it achieved the correct final state, not whether it followed some specific process; for complex workflows, break evaluation into discrete checkpoints where specific state changes should have occurred³. This lesson will also cover how to turn a fuzzy requirement into a measurable success criterion.

Lesson 3: Deterministic verifiers. How to pick and write checks that can produce pass/fail. Exact match, script comparison, test suites—what fits where, and one counterintuitive pitfall: an overly strict verifier will reject correct answers. The concrete verifier catalog and prioritization order will be in that lesson; we won't expand here.

Lesson 4: LLM as judge. Free-form text can't use string comparison; you have to ask a model to score it. How to write rubrics, how to constrain output format, whether to reason first or score first, and why the model doing the work shouldn't grade itself—we've already touched on this in the quiz earlier. Specific rubric design is in lesson 4.

Lesson 5: Evaluation sets. One check handles one task; a set of tasks makes an evaluation set. How to collect cases from real usage, how to fill in edge cases, what a holdout set does, and "how many is enough"—all answered in lesson 5; the answer might be smaller than you think.

Lesson 6: Build it yourself. Wire together the first five lessons: one evaluation task gets one harness loop, run it and produce a report; change one version of the prompt and see if the score moved.

Proportionality: Don't Wrap Every Small Thing in a Sign-Off Process

At this point, it's easy to swing to the other extreme: assuming every task needs tests, a judge, and an evaluation set. Not so.

Anthropic's original line is: the key to success, as with any LLM features, is measuring performance and iterating on implementations. To repeat: you should consider adding complexity only when it demonstrably improves outcomes⁴. The same article has a more specific route recommendation—start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short⁴.

Applied to verification, the decision criteria come down to a few lines:

• Will this task run repeatedly? A one-off script, ad hoc data processing, a three-minute job you plan to watch—setting up a sign-off mechanism is a net loss. Things that run repeatedly, get modified by others, or run while you're not around—worth it.
• Who bears the cost of an error? You fix a typo wrong, you roll it back yourself and it's done. You break billing logic, accounting bears the cost. The further downstream the cost and the harder it is to roll back, the more you should gate it up front.
• How much time do you spend verifying it now? If every time you have to manually open three pages and compare them, scripting that three-page comparison is the thing that most deserves automation—you're already paying this cost; you just haven't noticed.

One more case worth calling out separately: some checks you already have, you just haven't wired them to the agent. That test suite in the project, that lint command, that build script—they probably existed already. Writing them into the task description or making them into a tool costs almost nothing, but the nature of the session changes. This is the highest ROI step, and it's the starting point for the next few lessons in this course.

💻 Exercises

Recap

• Claude stops when the work looks done. Without a check it can run, "looks done" is the only signal available, and you become the verification loop: every mistake waits for you to notice it¹.
• The official docs name this gap: the trust-then-verify gap—Claude produces a plausible-looking implementation that doesn't handle edge cases. The paired fix's second half is equally important: if you can't verify it, don't ship it¹.
• The line between assertions and evidence is "can a second person re-run this exactly the same way." Have Claude show evidence—test output, the command it ran and what it returned, a screenshot of the result—not assertions of success. Reviewing evidence is faster than re-running the verification yourself, and it works for sessions you weren't watching¹.
• Agents are non-deterministic systems: even with the same starting conditions, they can generate varied responses²; even with identical prompts, decisions across runs aren't guaranteed to match³. So the traditional evaluation assumption "given input X, follow path Y, produce output Z" fails³—identical starting points can produce completely different but valid paths³.
• Errors in agent systems compound: one step failing can cause agents to explore entirely different trajectories, leading to unpredictable outcomes³. Autonomy brings higher costs and potential for compounding errors, so extensive testing in sandboxed environments with guardrails is recommended⁴.
• The way out is to give it a runnable check. With something that produces pass or fail, the loop closes on its own: do the work, run the check, read the result, iterate until it passes¹. The check can be a test suite, a build exit code, a linter, a script that diffs output against a fixture, or a browser screenshot compared against a design¹.
• During execution, let the agent gain "ground truth" from the environment at each step (tool results, code execution results) to assess progress, not from its own reasoning⁴.
• The LLM will potentially operate for many turns, and you must have some level of trust in its decision-making⁴—but that trust has to come from somewhere.
• Don't wrap every small thing in a full sign-off mechanism. You should consider adding complexity only when it demonstrably improves outcomes⁴; first check whether this task runs repeatedly, who bears the cost of errors, and how much time you spend manually verifying it now.
• The highest ROI step is often: that test suite, that lint command, that build script in your project already exists—you just haven't wired it to the agent yet.

>> Lesson 2: What to Verify: End State First, Process as Backstop