Lesson 2: Chain It, Route It: Chaining and Routing

Learning goals:

• Break a four-task prompt into a chain and articulate what you're trading and what you're gaining
• Install programmatic gates between chain stages so unqualified intermediate results stop where they should
• Decide when a task needs chaining, routing, both, or neither, and write the routing call as a single output-tightened cheap call

Prerequisites: Read Lesson 1, can hand-write a harness loop driven by stop_reason (Course 7 in this series), understand deterministic verifiers (Course 10 in this series) | Previous: << Lesson 1 | Next: Lesson 3 >>

One prompt doing four things—one will drop

You need to write help documentation for a new feature: read product requirements → draft an outline → write the full text based on the outline → check the document for deprecated terminology.

Your first version probably stuffs all four steps into one prompt and hands it to a harness loop. The first run looks fine. The problem shows up on the second run, and the third: this time the outline is good but the text is missing a section; next time the text is complete but it's mixed with "user groups" that were deprecated last version; then it rewrites the outline halfway through and delivers an outline that doesn't match the text.

These failures look different on the surface, but they share the same root: in a single call, the model has to simultaneously juggle understanding requirements, designing structure, generating text, and doing consistency checks. You can't control which one gets squeezed out, and you can't see it happening. Worse, you have no place to intervene—by the time you have the output, all four things are done, and "outline didn't meet spec" is buried inside the final draft.

Lesson 1 discussed the axis of "who holds the plan." This one-shot prompt hands the plan entirely to the model. The first thing this lesson does is take it back.

Chaining: trading latency for accuracy

The official definition of this pattern is just two sentences, and every word is load-bearing.

Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. You can add programmatic checks (see "gate" in the diagram below) on any intermediate steps to ensure that the process is still on track¹. When to use this workflow: This workflow is ideal for situations where the task can be easily and cleanly decomposed into fixed subtasks. The main goal is to trade off latency for higher accuracy, by making each LLM call an easier task¹.

"Making each LLM call an easier task"—that half-sentence gives you both diagnosis and cure. One call doing four things is a hard task; one call just drafting an outline and writing nothing is an easy task. Chaining doesn't reduce the work—it makes the work the model has to complete in each call simpler.

The cost is printed right on the price tag: latency. Each additional stage adds one complete round trip. Anthropic laid out this accounting at the start of the same article—agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense¹. "Slow and expensive" isn't an accidental side effect of chaining.

Requirements ──▶ Draft outline ──▶ Write full text ──▶ Check terms ──▶ Deliver

Each segment's input is the previous segment's output. If one link goes crooked, everything downstream follows.

Each stage is a complete harness loop

A stage in the chain isn't "one API call"—it's a complete harness loop. The same while loop you hand-wrote in Course 7 in this series: send messages, check stop_reason, if it's tool_use then execute the tool and send the result back, otherwise return the text.

This usage has a source. When Anthropic described how to evaluate agents, the recommended setup was exactly this shape: direct LLM API calls, simple agentic loops (while-loops wrapping alternating LLM API and tool calls), one loop per evaluation task, each evaluation agent given a single task prompt and your tools². That article was about evaluation, but the building block itself is general-purpose: one task, one loop, code-driven. Chaining is stringing these blocks together with code deciding the order.

The rest of the lessons use the same notation:

The entire chain is sequential code you can read at a glance:

Notice what's not in these lines: no room for "the model decides what to do next." The sequence is hardcoded, intermediate results outline and doc are ordinary script variables. The model remains autonomous within each stage (it can call tools as many times as it wants), but control between stages is in the code's hands. A side benefit: each stage's prompt can be strict about just one thing. The outline prompt demands only titles and forbids any body text; the writing prompt focuses on style and banned terms—these two sets of requirements would clash if packed into one prompt.

This shape appears in products too. Claude Code's official documentation recommends for multi-step workflows: have Claude use subagents sequentially, each completes its task and returns results to Claude, which then passes relevant context to the next subagent³. The difference lands on Lesson 1's axis—in the product form, Claude decides "what to pass"; when you write the script, that's your code.

Gates: moving Course 10 verifiers between stages

The last half-sentence in the definition is what chaining actually adds beyond "one big prompt": you can add programmatic checks on any intermediate steps to ensure that the process is still on track¹. The original text calls these checks "gate," and it's in quotation marks: (see "gate" in the diagram below)—one straight, one curly, exactly as in the source, not a typo here.

"Programmatic" is the key: it's code, not another model call, just a few if statements.

Course 10 in this series taught deterministic verifiers: when something can be judged right or wrong by code, don't spend money asking the model. That course installed verifiers at terminal state—after everything runs, check if the output is acceptable. Chaining offers the same check a new location: between stages.

Seven lines, not a single model call, same input always produces same judgment. It blocks exactly that "text mixed with deprecated terms" failure; a gate that counts how many chapter headings are in the outline is just as simple.

What to do when a gate fails is a design decision: stop and report the error (best when you're still tuning this chain, but the failure message must say which stage failed, otherwise you only know "it didn't work," not which stage prompt to fix), feed the failure reason back into the same stage's prompt and retry (with a retry cap), or log it and continue with a fallback value (only when this stage isn't load-bearing). Level 2 exercise uses the first approach.

This also settles the accounting from Course 6 in this series: work you delegate must have self-contained prompts—goal, output format, available tools, boundaries, all four written out. Each stage's task is a delegation prompt shaped exactly like that. These four elements have a precise primary source, which Lesson 4 will unpack item by item when it covers how orchestrators delegate.

Routing: classify first, then dispatch

Chaining handles "one task broken into steps." Another class of tasks has a completely different shape: what comes in isn't one thing, it's several kinds of things, each with its own handling.

The official definition: routing classifies an input and directs it to a specialized followup task. This workflow allows for separation of concerns, and building more specialized prompts. Without this workflow, optimizing for one kind of input can hurt performance on other inputs¹.

The last sentence is why routing exists. Suppose customer emails fall into three categories: refund, incident, billing. You use one prompt to handle everything. To handle refunds well, you add a line "first confirm order number and payment channel"; this rule is pure noise for incident emails, and the model will use it to ask someone reporting a white-screen page for their payment channel. You add another line "if it's an incident, don't ask for order number," and the prompt starts growing patches on top of patches.

When to use this workflow: routing works well for complex tasks where there are distinct categories that are better handled separately, and where classification can be handled accurately, either by an LLM or a more traditional classification model or algorithm¹. That last precondition isn't icing on the cake: if classification is wrong, it's wrong in a stealthy way—a refund email routed into the incident flow gets a conscientious troubleshooting response.

In code, routing is simpler than chaining:

Three things worth noticing.

Tighten the classification call's output to one word—Course 10 in this series used the same trick when discussing LLM judges: list the allowed values, say no explanation, tightening the output makes the parsing step deterministic. If it doesn't fit the table, go to fallback—that line LABELS.includes(label) ? label : 'other' isn't defensive pedantry. The model occasionally returns "I think it might be incident, or maybe something else," and then HANDLERS[that whole string] is undefined and the next line crashes. Leave one fallback branch and the classification uncertainty is contained in this one line.

The classifier doesn't have to be a model—the definition explicitly says traditional classification models or algorithms count too¹. If the email carries a fixed order-number format or comes from a dedicated form entry point, one regex is enough and much faster.

After dispatch, each handler can be anything: a harness loop, a chain, even a segment of completely model-free code.

Two routing variants in current API vocabulary

The routing definition above comes from the late-2024 patterns article, which carries a banner saying its tooling-ecosystem descriptions are outdated. So it's worth checking: is this pattern still alive in current first-party vocabulary? Yes, and it's called out by name. Claude platform's multi-agent orchestration documentation has two entries that are routing:

• Specialization: Route to agents with domain-focused system prompts and tools, such as a security agent or a documentation agent, rather than loading a single agent with every capability⁴. This is the official phrasing for the HANDLERS table.
• Escalation: Consult a more capable agent or model for a subset of complex subtasks⁴.

The second one deserves separate mention: it dispatches by difficulty, not topic. The classifier doesn't judge "is this refund or incident" but "can this email be handled by my cheap tier?" This is harder to judge accurately than topic classification, so the escalation path has a more stable writing: run the cheap tier first, if the output fails the gate then escalate—swap a hard-to-judge classification problem for a checkable verification problem.

Knowing when not to split

Each link in the chain adds latency. This isn't an unoptimized implementation, it's the price the official definition sets: trading latency for higher accuracy¹. The user waits for the sum of every segment. If the user is synchronously waiting in a UI for results, before adding another link to the chain, consider whether the person is still there.

When there's only one category, routing is pure overhead. Routing's benefit comes from separation of concerns¹. If input really only has one type, you paid for a classification call's cost and latency and got nothing in return, plus one more chance to misclassify.

When the task can't be cleanly split, don't force it. The condition "easily and cleanly decomposed into fixed subtasks" has teeth¹. A draft that needs to look at the whole picture back and forth to revise well, if you split it into "revise structure first, then revise wording," the second stage doesn't have access to the reasons the first stage didn't write down, and will only revise based on literal text. In this case one loop, one context is actually better—this is exactly the use of Lesson 1's reverse conditions.

When unsure, measure first. Anthropic said this twice: consider adding complexity only when it demonstrably improves outcomes¹. The evaluation track from Course 10 in this series is built for this: run the single-prompt version and get a score, run the split version and get another score, see how much difference and whether it's worth those extra seconds of latency—that's the evidence you can take into an argument with a colleague.

Finally, mark the boundary. Chaining and routing are both fixed-shape orchestration: how many stages the chain has, which categories the router has, all decided when you write the code. Running multiple stages simultaneously and then aggregating is Lesson 3's parallelization; not even knowing how many subtasks there are until you see the input is Lesson 4's orchestrator-workers.

💻 Exercises

Recap

• Chaining decomposes a task into a sequence of steps where each call processes the previous output, making each call an easier task¹; it's priced explicitly: the main goal is to trade off latency for higher accuracy¹.
• A stage in the chain is a complete harness loop, not one API call—the setup Anthropic recommended for evaluation is exactly this building block: one task, one while loop, code directly calling API².
• The positions added after splitting are the key: you can add programmatic checks on any intermediate steps to ensure that the process is still on track¹. This is Course 10's deterministic verifier moved from a different installation location—from terminal state to between stages; in product form it looks like subagents executing sequentially, each completes then the upper layer passes relevant context to the next³.
• Routing classifies input and dispatches to specialized followup tasks, gaining separation of concerns and more specialized prompts; without it, optimizing for one kind of input can hurt performance on other inputs¹. Precondition: clear categories and classification itself can be handled accurately, either LLM or traditional classification algorithms¹.
• Tighten the classification call's output to one label, and leave one fallback branch to catch answers that don't fit the table.
• This pattern is alive in current first-party vocabulary: dispatching by domain to agents with dedicated prompts and tools is called specialization, consulting a more capable agent or model for a subset of complex subtasks is called escalation⁴—the latter is routing dispatching by difficulty.
• Don't force it: when there's only one category routing is pure overhead, when the task can't be cleanly split forcing it will lose information between stages¹. When unsure, measure first—complexity has to pass the "demonstrably improves outcomes" threshold¹.

>> Lesson 3: Parallelization: Sectioning and Voting