Lesson 5: The Review Loop, and Composing Patterns into a Graph

Learning goals:

• Implement a review-and-refine loop (generate a draft, evaluate it, revise based on feedback), and use two decision criteria to determine whether this loop is worth building
• Write stopping conditions smarter than "max N rounds," and place deterministic checks before the judge
• Compose five patterns into what this lesson calls a "graph," and document in your own materials that this visual system is a custom metaphor anchored to a specific primary source quote

Prerequisites: Read Lessons 1–4 (who holds the plan, chaining and routing, parallelization, orchestrator-workers), can hand-write a harness loop driven by stop_reason | Previous: << Lesson 4 | Next: Lesson 6 >>

The draft that's always one step short

You ask an agent to write a database migration plan. The first draft comes back looking reasonable: background, steps, time window all present. But you spot two holes in one glance—the rollback section just says "roll back if necessary," and there's no risk rating anywhere. You type two lines of feedback pointing these out, the second draft comes back, both holes are filled, and the whole thing jumps up a quality tier.

You've repeated this process dozens of times. Every time it's the same: output falls short, human provides two sentences of feedback, output gets noticeably better.

The problem isn't that the model writes poorly. The problem is that your two sentences of feedback aren't hard to produce. "Rollback steps must include specific commands," "Each step needs a risk rating"—these are things a checklist could cover. If you can articulate it clearly, the model can probably articulate it too. So why does it have to be you saying it every time?

This shape is worth writing as a loop.

Review-and-refine: Writing "revise once more" into control flow

The primary source defines it in one sentence: one LLM call generates a response while another provides evaluation and feedback in a loop¹.

The first four lessons' four patterns each have their own topology: chaining decomposes a task into a sequence of steps, with each step processing the previous one's output¹; routing classifies and then dispatches to specialized followup tasks¹; parallelization runs things simultaneously and aggregates results programmatically¹; orchestrator-workers has a central LLM dynamically break down tasks, delegate to workers, and synthesize results¹. They all share one trait: data flows forward. The review loop is the first pattern with a back edge—output loops back into the generation node.

What does it look like in a product? Claude Code's dynamic workflow documentation gives one plain-English description: run a checker, fix what failed, and repeat until it passes or stops making progress². Another description covers a different use of the same division of labor: have independent agents adversarially review each other's findings before they're reported². That's a one-time cross-review before reporting, with no back edge and no iteration—lumping it into the review loop is this lesson's categorization, not the original text describing the same topology.

One concept, three names

We need to build an explicit bridge here, or you'll think you're learning three different things.

This series' Course 6, which teaches multi-agent collaboration, calls this "one does the work, one critiques" division producer-reviewer. That's our own teaching vocabulary. In primary materials, Claude Code's workflow documentation describes its product use in one sentence: have independent agents adversarially review each other's findings (this lesson's shorthand for that sentence is "adversarial cross-review"). The same shape has two names in primary sources: Anthropic's pattern reference calls it evaluator-optimizer¹; Claude Code's workflow documentation doesn't give it a name, just describes the use—that sentence about adversarially reviewing each other's findings².

Three names, one shape. The difference is which angle you're looking from: when discussing collaboration roles you see two actors, when discussing orchestration patterns you see a back edge, when discussing product capabilities you see a reusable quality technique.

There's one more division of labor to clarify. This series' Course 10 spends an entire course teaching you how to be a good judge: how to write rubrics, how to constrain the judge's output format, why the judge needs its own independent context, why the worker shouldn't also be the judge. That course teaches the quality of the judge itself. This lesson doesn't repeat those topics. This lesson teaches how to wire the judge into control flow—where in the loop it sits, when it runs, how many rounds it runs, when it stops.

When this loop is worth building

The primary source's applicability criteria: this workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value; the two signs of good fit are, first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback¹.

You've seen this quote before. This series' Course 10 quoted this exact sentence when answering the question "is a review-revise loop worth building." Same criteria, reframed in an orchestration context—except this time you're implementing the answer as a loop in control flow.

Broken out, these two signs each guard against different failure modes:

The first sign guards against "revision doesn't help." Some tasks won't get better in a second draft no matter how clearly you state the feedback—because the problem is missing input data or a vaguely defined task, not the wording of the output. In this case, building a loop just means you're paying twice to get two equally unusable versions. The validation method is crude but effective: do it manually three times yourself. How many of those three were "clearly better after human feedback"? If two out of three were "feedback didn't help," don't build the loop.

The second sign guards against "the judge can't give that kind of feedback." Even if human feedback works, you still have to ask: can the model itself provide the same kind of feedback? If your feedback depends on things only you know (what this customer complained about last quarter, what legal verbally briefed last week), the model doesn't have that information, so the feedback it gives will be something else entirely. In that case, either feed that information into the judge's prompt—turn it into criteria the model can evaluate—or accept that this step needs a human.

There's a precondition that kicks in even earlier than these two signs: evaluation criteria must be clear. When criteria aren't clear, the loop reliably produces a specific kind of failure—the judge gives feedback pointing in different or even contradictory directions each round, output ping-pongs between two versions, rounds burn out, and the final draft is worse than the first. This isn't the loop's fault. It's that the criteria haven't been defined yet.

Deterministic checks come before the judge

The primary source's definitional distinction between deterministic and non-deterministic systems: deterministic systems produce the same output every time given identical inputs, while non-deterministic systems—like agents—can generate varied responses even with the same starting conditions³.

Judges are non-deterministic. Every rule that "code can decide definitively" handed to a judge means you're using something that might give different results each time to evaluate something that should give the same result every time—while also paying for an extra model call.

This series' Course 10 calls this discipline layered scoring: use code for what code can decide, only hand to the model what code can't. This lesson copies it straight into the loop's node ordering. The quote from Lesson 2 still applies here—you can add programmatic checks at any intermediate step to ensure the process is still on track¹. Every draft in the loop is an intermediate step.

Applied to the migration plan example: "Does each step have a corresponding rollback command?" can be decided definitively with regex or structured parsing—that's a gate. "Are the rollback commands written credibly?" needs a judge. The former failing doesn't even wake up the judge; just tell the writer which steps are missing and move on.

The loop's code skeleton

A few details worth calling out individually.

runAgent is a complete harness loop. This hasn't changed since Lesson 2: every await runAgent(...) in this script has, behind it, the stop_reason-driven loop from this series' Course 7 running. This is just another layer of code-written control flow wrapped around the loop.

These reason values are different outcomes, don't collapse them into a boolean (real systems often need to subdivide further—for example, gate repeatedly failing gets its own bucket). passed can be delivered directly; max-rounds means rounds burned out without passing, likely needs human handoff; no-progress means the model got stuck, burning more money won't make it better. These three outcomes should be three separate lines in your observability data—the logging approach Course 11 of this series teaches should land on this reason field here.

Gate failure also counts as a round. Before the continue, rounds has already incremented. This is intentional: repeatedly failing the gate means the writer's prompt has a problem, letting it retry indefinitely just burns money on the same hole.

You need both kinds of stopping conditions. The primary source discussing agent loops says: the task often terminates upon completion, but it's also common to include stopping conditions (such as a maximum number of iterations) to maintain control¹. That's where "max N rounds" comes from—it's a fuse, guaranteeing this code stops under any circumstance. The "no further progress" stopping mechanism comes from elsewhere: run a checker, fix what failed, and repeat until it passes or stops making progress². It's smarter than the fuse because it watches whether this round beat the last round, not how many rounds have run.

Score not rising means exit—that's the easiest implementation, but not the only one. If your judge doesn't output a score, you can switch to watching whether the count of failing items decreased; if the task itself is high-variance, you can switch to "exit only after two consecutive rounds without improvement" with a stalled counter. Which one you pick depends on how stable your judge is, not which one sounds sophisticated. (Note the bestDraft in the skeleton: exiting when score doesn't rise presumes you're always holding onto the highest-scoring draft; tracking only bestScore without bestDraft means the no-progress and max-rounds exits will hand out the worse current version.)

Composing patterns: This lesson calls it a "graph"

Five patterns, all accounted for. The next question is how to arrange them together.

Official position first: these building blocks aren't prescriptive, they're common patterns that developers can shape and combine to fit different use cases; the key to success, as with any LLM features, is measuring performance and iterating on implementations¹.

In other words, for "how to compose," the primary source gives permission, not a recipe. You write the recipe yourself.

An honest declaration about graph terminology

This lesson's use of "graph / nodes / edges" throughout is this lesson's own engineering metaphor, not official terminology (the preceding sections have already used "node" and "back edge" with this meaning).

Copy this sentence into your own architecture documentation. The words "graph," "node," "edge," "DAG," "state machine" appear zero times across all primary sources this lesson cites. The primary source vocabulary is workflows, patterns, orchestrator-workers, fan out—it's discussing a pattern catalog, not topological structure.

We still need to use the word "graph," because when five patterns sit together you need a language to talk about them clearly, and "graph" is the least-effort option. But this metaphor must have an anchor point, or it's just made-up jargon. The anchor is this sentence that actually exists in primary materials: the workflow script itself holds the loop, branching, and intermediate results, while the model's context holds only the final answer².

That sentence already names all three elements of a graph: loop (back edge), branching (fork point), intermediate results (state). All we're doing is giving each a name.

Drawing conventions

In this lesson's visual system:

• Node = one runAgent loop, or a piece of pure code (gate, classification, aggregation, batching). Labeling each node's type is the most valuable action when drawing—it forces you to answer "does this step actually need the model?"
• Edge = "whose output feeds whom." An edge isn't a data structure, just the next line of code reading the previous line's variable.
• State = script variables. The primary anchor has a sentence here too: intermediate results stay in script variables instead of landing in the model's context². There's no official concept of "a state object passed between nodes"—that's a phrase we borrowed from other domains; this lesson doesn't build that abstraction, just pass whatever variables you need.

Five patterns in this visual system's shapes

Chaining       A ──> B ──> C                    A straight line
Routing           ┌──> B1               A ─┼──> B2                       One fork point                  └──> B3
Parallelization   ┌──> W1 ──┐               A ─┼──> W2 ──┼──> merge          A fan: fan out, then merge                  └──> W3 ──┘
Orchestrator-     ┌──> W? ──┐                   Fork point is dynamic: how many  workers      A ─┼──> W? ──┼──> merge          edges, what each does, decided by A                  └──> W? ──┘                   after seeing input
Review loop    A ──> J ──┐                      One loop with a back edge               ^         │               └───no────┘

That fourth shape's annotation is worth re-reading. Parallelization and orchestrator-workers are topographically similar in shape, with the key difference being that subtasks aren't predefined but determined by the orchestrator based on specific input¹. Drawn on paper, the difference is "I drew three edges" versus "A drew three edges"—indistinguishable on paper, but very different in code.

A composition example

Stringing together routing, fan-out, merge, and review loop:

[classify] ─┬─ simple ──> [direct answer] ──────────────────────> deliver  pure code │ (keyword + └─ complex ──┬─> [worker1] ─┐   rules)                ├─> [worker2] ─┼─> {merge} ─> [draft] <──────┐                         └─> [worker3] ─┘              runAgent │       │                                                           v      │       │                                                        {gate} ─no────────┤                                                           │ pass  │       │                                                           v      │       │                                                        [review] ─no──────┘                                                           │ yes   runAgent                                                           v                                                        deliver
Node types: [ ] = runAgent loop    { } = pure codeEdge = whose output feeds whom. {gate} is deterministic check, comes before [review]; gate fail or review no both kick back to [draft].[classify] is drawn as a model loop rather than {pure code} because the refund/tech/complaint boundary is fuzzy—when boundaries are clear, swap in a traditional classifier (this is layered scoring: use code first when code can decide).

Lesson 6 implements a variant of this graph: that batch of tickets happens to have acceptance criteria that can all be written as rules, so the [review] layer degrades to a {gate}, and fan-out switches from "one complex item to three workers" to "one batch of tickets, each dispatched to one handler." Which parts changed and why—Lesson 6's opening lists them point by point. Look at the shape here first, code waits until the next lesson.

Engineering benefits from composition

Moving control flow into code delivers more than just "understandable." A few benefits have primary backing:

Step-by-step tracking brings recoverability. The runtime tracks each agent's result as the run progresses, which is what makes a run resumable within the same session². Translated into this lesson's visual system: every node in the graph is naturally a checkpoint location—node finishes, result lands in a script variable, that variable is the "where we got to" record. The checkpoint design taught in Course 9 of this series doesn't need a separate foundation here; node boundaries are natural landing points.

Fine-grained fan-out preserves more progress. The primary source's words: a workflow that fans work out across many small agents preserves more progress than one long agent². One forty-minute long agent crashes, forty minutes gone; forty one-minute small nodes and one crashes, you lose one minute and you know which minute.

Repeatable quality techniques become reusable. Moving the plan into code also lets a workflow apply a repeatable quality pattern, not just run more agents: it can have independent agents adversarially review each other's findings before they're reported, or draft a plan from several angles and weigh them against each other, so you get a more trustworthy result than a single pass². The key word here is "repeatable"—doing one cross-review manually is an operation, writing it into a script is a capability.

Deterministic guardrails wrap non-deterministic agents. Anthropic's multi-agent research system retrospective states: they combine the adaptability of AI agents built on Claude with deterministic safeguards like retry logic and regular checkpoints⁴. Translated into this lesson's visual system: the graph's skeleton is deterministic (who calls whom, when to stop, which edge to take on failure), node interiors are non-deterministic. This layering isn't aesthetic preference, it's a prerequisite for making the system operable.

Composition discipline: Every added layer must pass one gate

Benefits stated, now constraints.

Every added layer must pass the "measurable improvement" gate. The primary source says the same thing in two places, with the second explicitly flagged as a reiteration: you should consider adding complexity only when it demonstrably improves outcomes¹. This is especially critical for this lesson—five patterns laid out in front of you, the easiest mistake is using all of them. Adding one node means one more model call, one more place that can fail, one more thing to troubleshoot. Before adding, ask: remove it, do metrics drop? Can't answer means you haven't measured yet.

Node-level retry and timeout are engineering practice, not official design. Primary sources mention this only as a subordinate clause (deterministic safeguards like retry logic and regular checkpoints⁴). So the following is written as engineering practice; you won't find their endorsement in any primary documentation: wrap every runAgent node in a timeout, after timeout either retry or mark this node failed and continue; retry count depends on the node's nature (read-only retrieval nodes can retry several times, nodes with side effects ideally don't auto-retry even once); when a node fails, distinguish "this edge can be skipped" from "the entire graph must stop," don't let one optional node's failure drag down the whole run. These are ordinary distributed-systems common sense, just applied to agents—don't treat them as anything new.

Depth is bounded. The product-level reference is right there: by default, a subagent can spawn subagents of its own, up to three layers below the main conversation⁵. Three layers isn't a threshold this lesson invented, but the message is clear—nesting depth in real products isn't unlimited, someone seriously thought about where to stop. Your graph should have a similar answer. If your drawn graph has five layers of nesting, suspect the task is decomposed too finely first, don't go thinking about how to support deeper.

The graph isn't the goal, it's a description of task shape

One failure mode is especially worth preventing at this course's end: pick a cool topology first, then find tasks to stuff into it.

The order should reverse. Draw the task's own dependency shape first—which steps must queue (previous step's output is next step's input), which steps don't affect each other (doesn't matter who runs first), which step needs to see input before knowing how many pieces to split into, which step's output needs someone to critique before it's trustworthy. That drawing finished, which patterns to use is basically decided: queuing places are chains, mutually independent places are fans, see-then-decide places are orchestrators, needs-critique places are loops.

Patterns are names for task shapes, not a menu you can pick arbitrarily from.

And above that discipline is one that kicks in even earlier: find the simplest solution possible, and only increase complexity when needed¹. This sentence appeared in Lesson 1, and at this lesson's end it's still the same sentence. After learning five patterns, "one LLM call is enough" remains a completely valid answer—primary sources themselves say that for many applications, optimizing single LLM calls with retrieval and in-context examples is usually enough¹.

Complete, runnable composition code is in Lesson 6. This lesson stops here. What you have now: five patterns, one visual system, and a list of when not to use them.

💻 Exercises

Recap

• Review-and-refine is one LLM call generating a response while another provides evaluation and feedback in a loop¹; its product form is "run a checker, fix what failed, repeat until it passes or stops making progress"², adversarial cross-review is another use of the same division of labor (single cross-review, no back edge), lumping it into review loop is this lesson's categorization². Course 6 of this series calls it producer-reviewer, that's our teaching vocabulary, primary vocabulary is evaluator-optimizer¹.
• Worth building depends on two signs: LLM responses can be demonstrably improved when a human articulates their feedback, and the LLM can also provide such feedback; it's particularly effective when evaluation criteria are clear and iterative refinement provides measurable value¹. When criteria aren't clear, define criteria first, don't build loop first.
• Stopping conditions aren't just one kind: stopping conditions like maximum iterations are used to maintain control¹, "no further progress" is another more cost-effective stopping mechanism²; three outcomes (pass / rounds burned / no progress) map to three different downstream actions, don't collapse into a boolean. Deterministic gates come before judges.
• Five patterns can be composed: these building blocks aren't prescriptive, they're common patterns developers can shape and combine to fit different use cases, the key to success is measuring performance and iterating on implementations¹.
• "Graph / nodes / edges" is this lesson's own visual system, not official terminology; its primary anchor is just one sentence—the workflow script itself holds the loop, branching, and intermediate results, while the model's context holds only the final answer², plus intermediate results stay in script variables². When using this vocabulary in your own docs, include this declaration with it.
• Composition benefits are documented: the runtime tracks each agent's result as the run progresses, which is what makes a run resumable within the same session²; a workflow that fans work out across many small agents preserves more progress than one long agent²; moving the plan into code also lets a workflow apply repeatable quality techniques (adversarial cross-review, draft from multiple angles then weigh)²; deterministic safeguards (retry logic and regular checkpoints) wrap non-deterministic agents⁴.
• Constraints are equally clear: you should consider adding complexity only when it demonstrably improves outcomes¹; node-level retry and timeout are ordinary engineering practice, primary sources offer only one subordinate clause⁴; depth is also bounded, product-level reference is subagents nest up to three layers below the main conversation⁵; simplest solution first¹.

>> Lesson 6: Hands-On: Upgrading Your Harness into a Small Graph