Lesson 3: Parallelization: Sectioning and Voting

Learning goals:

• Distinguish between the two variants of parallelization—sectioning and voting—understand what each solves, and respect the boundary drawn by "outputs aggregated programmatically"
• Use Promise.all and a custom concurrency pool to implement sectioning, ensuring aggregation passes references rather than payloads
• Calculate the three costs of fan-out (results flooding context, real product concurrency ceilings, voting's N× token multiplier) and use them to decide whether a proposal should parallelize

Prerequisites: Completed Lessons 1 and 2, have the runAgent() wrapper from Lesson 2 | Previous: << Lesson 2: Chain It, Route It: Chaining and Routing | Next: Lesson 4 >>

Twelve docs, one chain, an hour in the queue

The chain from Lesson 2 now works: outline → gate → draft → gate → check terminology. Swap in review-focused prompts—extract key points, suggest revisions, verify terms—and the chain's shape doesn't change. Run it on one document: six or seven minutes.

Then the product team drops a directory on your desk: 12 documents, each needs the same review pass.

You write a for loop, start it, and go make coffee. An hour later you're back. The log stopped at document 9. Document 10 is extracting key points.

For that hour, the machine spent most of its time waiting. Waiting for document 1's API response before sending document 2's request. Waiting for document 2 to finish all four stages before document 3 gets its turn. Ask a practical question: does document 3's review conclusion depend on a single word from document 2's result?

No. They're 12 independent documents. Their review reports don't care who finishes first. In chaining, the wait has a reason—the next step's input is the previous step's output. Here there's no such reason. These 12 runs are only queued because a for loop put them in line.

Course 6 in this series already covered fan-out-aggregate collaboration patterns and how multi-perspective voting works¹. This lesson turns it into code and settles the accounts: fan-out isn't free. The speed is real, and so is the cost.

Definition: can run simultaneously, outputs aggregated programmatically

Start with the original wording. LLMs can sometimes work simultaneously on a task and have their outputs aggregated programmatically. This workflow is parallelization, with two key variations¹:

• Sectioning: Breaking a task into independent subtasks run in parallel¹. Reviewing 12 documents is sectioning.
• Voting: Running the same task multiple times to get diverse outputs¹. Having three perspectives each evaluate the same copy is voting.

When to use it: when the divided subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed for higher confidence results¹. There's an easy-to-skip but valuable addition—for complex tasks with multiple considerations, LLMs generally perform better when each consideration is handled by a separate LLM call, allowing focused attention on each specific aspect¹. Translation: sectioning and voting aren't just time-savers. Cramming "legal, security, brand" into one prompt versus having three calls each watch one thing produces different quality.

One more half-sentence to nail down: outputs aggregated programmatically. After fanned-out results return, your code does the judging, filtering, and summarizing—not another model call to read all 12 reports and write a summary. Having the model aggregate is a different pattern. Lesson 4's orchestrator does exactly that work. Draw the line clearly here. Current Claude platform docs list Parallelization under multi-agent orchestration: fan out independent subtasks simultaneously (searching multiple sources, analyzing separate files) and have the coordinator synthesize the results²—note that in that version the coordinator does aggregation, while this lesson writes the programmatic aggregation version¹. Same word, who does the aggregation is two different things.

Sectioning: swap the for loop for Promise.all

Serial version looks like this, total time is the sum of all 12:

Sectioning version changes one line, total time approaches the slowest one:

runAgent() still sits at the same wrapper position from Lesson 2—one complete harness loop, internally branching on stop_reason. This lesson's stub responses all complete in one turn (end_turn), so the version in the final script omits the tool branch as a simplification. When connecting to a real client or needing tools, bring back Lesson 2's tool-dispatching version unchanged. Parallelization doesn't alter any line of this loop itself. It just stops making these loops queue.

Aggregation happens in the next line, done by this code:

These three lines contain no second model call. filter, reduce, a threshold check—all deterministic code. Same 12 reports in, same one-line conclusion out every time. That's the benefit of keeping aggregation in code: the 12 fan-out calls are non-deterministic, the merge step is deterministic. When something breaks, you know which side to suspect.

Promise.all has a temperament to know up front: if any one promise rejects, the whole await rejects, and even if the other 11 finished, you can't get their results. Review 12 documents, document 7 hits a 500 and the whole batch is wasted—the other 11 ran for nothing. That cost is unreasonable. Either switch to Promise.allSettled, or like this lesson's exercise, wrap each worker in try/catch to collect failures as records—every fan-out path should be able to fail independently.

Why parallelization isn't just about speed

If parallelization were only "same thing done sooner," it would be a performance trick, not worth its own lesson. The real reason sits on the context side.

Anthropic's retrospective on their multi-agent research system is blunt: the essence of search is compression—distilling insights from a vast corpus. Subagents facilitate compression by operating in parallel with their own context windows, exploring different aspects of the question simultaneously before condensing the most important tokens for the lead research agent. Each subagent also provides separation of concerns—distinct tools, prompts, and exploration trajectories—which reduces path dependency and enables thorough, independent investigations³.

Break those two sentences apart. Fan-out buys at least three things:

1. Window capacity. They wrote this architectural judgment as a conclusion: distributing work across agents with separate context windows adds capacity for parallel reasoning³. Lesson 1 covered this: what really hits the ceiling isn't window size, it's "one loop" as a shape. Fan-out works around single-window limits not by stretching the window but by opening several.
2. Separation of concerns. Three subagents carrying different tools and prompts naturally won't pollute each other.
3. Reduced path dependency. In one loop, step 3's judgment gets biased by step 2's phrasing. Three independent trajectories don't share the same bias.

Current platform docs offer the same direction: multiple agents can act in parallel with their own isolated context, which helps improve output quality and can also improve time to completion². Note quality comes first.

On the speed side they gave a number, with context that must be copied together: their early agents executed sequential searches, which was painfully slow. For speed, they introduced two kinds of parallelization: (1) the lead agent spins up 3-5 subagents in parallel rather than serially; (2) the subagents use 3+ tools in parallel. These changes cut research time by up to 90% for complex queries³.

This number must be used with its three qualifiers: it's a latency number, not a quality number; it's limited to complex queries (simple queries don't have much to parallelize); it comes from their own system. How much you save by swapping for for Promise.all depends on how much of your subtasks are truly independent, how slow each path is, and where concurrency gets bottlenecked—the rest of this lesson is about that.

First cost: aggregation eats back the context you saved

When fanning out, everyone watches "how many paths run at once." Crashes usually happen on the way back.

Claude Code's subagent docs put this cost on the table: when subagents complete, their results return to your main conversation. Running many subagents that each return detailed results can consume significant context⁴. Subagents exist to protect main conversation context—keeping exploration and implementation out of your main conversation⁴—but once what returns is too heavy, protection reverses.

The same retrospective gave a remedy, and gave it specifically: rather than requiring subagents to communicate everything through the lead agent, implement artifact systems where specialized agents can create outputs that persist independently. Subagents call tools to store their work in external systems, then pass lightweight references back to the coordinator³.

Translate that into a mantra for writing code: pass references, not payloads.

How big is the difference? This lesson's exercise script gives you a real number: 8 outputs on disk total 1,668 bytes, what returns to aggregation is only 643 bytes, and this ratio widens rapidly as documents grow—reports ten times longer, what passes back is still a one-line summary plus a path. Whoever needs the full text, read it from the path.

This path buys other things incidentally. Workflow docs mention that the runtime tracks each agent's result as the run progresses, which is what makes a run resumable within the same session. A workflow that fans work out across many small agents therefore preserves more progress than one long agent⁵. Outputs land outside, leaving a record line by line—the record-keeping part is the same principle as observability in Course 11 of this series. The "interrupted mid-run doesn't start from scratch" part is Course 9's territory, "making long tasks survive interruption."

Second cost: concurrency is never unbounded

The moment you write Promise.all(docs.map(...)), you're actually saying "concurrency = array length." Array is 12, fine. Array is 200, that's another story.

Look at three real products' ceilings:

• Claude Code: by default, when 20 subagents are running in a session, spawning another with the Agent tool fails with Concurrent subagent limit reached, and the error message explicitly tells Claude not to retry⁴.
• Claude Code workflow runtime: up to 16 concurrent agents, fewer when Claude Code has fewer CPUs available (including inside a CPU-limited container)⁵; 1,000 agents total per run⁵.
• Managed Agents: a maximum of 25 concurrent threads is supported. The coordinator can call multiple copies of a single agent in the roster, creating multiple threads associated with one agent².

Three different teams, three different implementations, all set ceilings, and the numbers aren't even large. This fact itself is teaching material: unbounded fan-out is an accident, not an optimization. (Lesson 4 will cover a real crash case—early agents would spawn 50 subagents for a simple query³. By then you'll find that "who decides how many to spawn" is trickier than "what's the ceiling.")

The cheapest throttling is batching:

Works, but has a bucket effect: each batch waits for its slowest to finish before starting the next. Of three documents, one is especially long, the other two paths just wait.

A concurrency pool doesn't have this problem—fix N "lanes," each lane grabs the next item from a shared cursor as soon as it finishes current work, always N in flight:

Ten-ish lines, no dependencies. cursor++ is safe in single-threaded JavaScript—synchronous code between two awaits can't be interrupted. No risk of two lanes grabbing the same index. In the exercise you'll add try/catch so single-path failure doesn't drag down the whole batch.

What should limit be? No universal answer. It's the intersection of your API quota, downstream service capacity, and single-path duration. But filling in a specific number versus not filling one is two kinds of engineering.

Third cost: voting pays N× tokens

Voting's definition is one sentence: running the same task multiple times to get diverse outputs¹. Code is short:

Aggregation here is still done by code—filter plus a threshold. How many votes set the threshold is a product decision, hardcoded, changeable anytime, auditable. It shouldn't be left to the model's improvisation. High-risk scenarios can tune the threshold to "one veto," low-risk scenarios can require all three votes to block.

The accounting is straightforward: vote N times, pay N× tokens. Put this money alongside Lesson 1's multiplier—by their data, agents use about 4× the tokens of chat interactions, multi-agent systems about 15×. Multi-agent systems therefore need tasks valuable enough to cover the cost of this performance boost³. Three-perspective voting's price tag is that 4× for single-agent times 3 more. Don't multiply 15× by 3—that 15× already includes fan-out accounting.

So voting isn't "run a few more times for peace of mind." It must buy something concrete. Workflow docs are clearer: moving the plan into code also lets a workflow apply a repeatable quality pattern, not just run more agents—it can have independent agents adversarially review each other's findings before they're reported, or draft a plan from several angles and weigh them against each other, so you get a more trustworthy result than a single pass⁵.

"Independent agents review each other" is the same rule as Course 10 in this series: the worker doesn't judge their own work. Having the model self-check in the same context, it will mostly defend what it just output. Switch to an independent context path, switch prompts, then it might actually catch problems. What voting and peer review buy isn't "majority," it's independence.

Note a boundary: three calls to one model aren't three independent judges. They share the same training biases. Voting can filter sampling noise and single-pass attention gaps. It can't filter systematic biases. Don't treat it as a mechanism that produces truth just by voting.

Measure: independence is a prerequisite, not optional

All this lesson's benefits rest on one prerequisite that's appeared repeatedly and is worth pulling out: subtasks must truly not depend on each other.

Claude Code docs, when discussing multiple subagents investigating simultaneously, specifically add a sentence: each subagent explores its area independently, then Claude synthesizes the findings. This works best when the research paths don't depend on each other⁴. The reverse condition is written in the multi-agent retrospective: some domains require all agents to share the same context or involve many dependencies between agents, and these domains are not a good fit for multi-agent systems today. For instance, most coding tasks involve fewer truly parallelizable tasks than research, and LLM agents are not yet great at coordinating and delegating to other agents in real time³.

To judge whether a proposal should parallelize, ask one question: Does path 2 need to wait for path 1's conclusion to know what to do?

• Need to wait → this isn't parallelization's shape. Previous step's output is next step's input—that's Lesson 2's chaining.
• Don't need to wait → sectioning.
• Same thing, want several independent judgments → voting.

The first case is easiest to blur: there's a dependency but you force fan-out because "it'll probably work out." Result is several agent paths each writing their own conclusions, unaware of others. At aggregation you manually smooth contradictions—the time saved is all spent smoothing, plus you paid extra tokens.

Boundary: this lesson's decomposition is predefined

Pin down one word—it's Lesson 4's entrance.

In all this lesson's examples, who defined the subtasks? You did. Twelve documents, you read them from the directory. Three perspectives, you hardcoded them in an array. Before the code runs, how many paths fan out and what each does is all settled. This is called predefined decomposition.

Its opposite: the model decides how many parts and what each does. The original source treats this difference as the key watershed between two patterns—in the orchestrator-workers workflow, a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results¹. Though topographically similar to parallelization, the key difference is its flexibility—subtasks aren't pre-defined, but determined by the orchestrator based on the specific input¹.

So the boundary between the two isn't "how many paths run at once," but "who wrote that array." Array is yours, it's this lesson. Array is generated by the model on the spot, it's the next lesson. Last lesson's routing already handed one decision to the model (which branch). Next lesson hands over a bigger piece: decomposition itself.

💻 Exercises

Recap

• Parallelization's definition is "LLMs can sometimes work simultaneously on a task and have their outputs aggregated programmatically," two variants are sectioning (breaking a task into independent subtasks run in parallel) and voting (running the same task multiple times to get diverse outputs)¹
• Its conditions are subtasks can parallelize for speed, or need multiple perspectives and attempts for higher confidence. For complex tasks with multiple considerations, handling each with a separate call for focused attention generally performs better¹
• Fan-out buys more than speed: subagents operating in parallel with their own context windows explore and compress the most important tokens back, also bringing separation of concerns (different tools, prompts, exploration trajectories) and less path dependency³. Distributing work across agents with separate context windows adds capacity for parallel reasoning³
• The speedup number comes with context: their Research system introduced two-level parallelization (lead spins 3-5 subagents in parallel, subagents use 3+ tools in parallel), cutting research time by up to 90% for complex queries³—this is a latency number from their own system, not a quality number
• Aggregation is the first cost: subagent results return to main conversation, many subagents each returning detailed results consumes significant context⁴. Solution is artifact systems—subagents store output in external systems, pass only lightweight references back to coordinator³
• Concurrency is never unbounded: Claude Code defaults to 20 concurrent subagents, fails with explicit "don't retry" when exceeded⁴. Workflow runtime supports up to 16 concurrent agents (fewer when CPU-limited), capped at 1,000 per run⁵. Managed Agents maxes at 25 concurrent threads²
• Voting's cost is N× tokens, must be placed in that multiplier together—by their data, agents are about 4× chat, multi-agent systems about 15× chat. Task value must be high enough to cover it³. What it should buy is a repeatable quality pattern, like independent agents adversarially reviewing, or drafting from several angles then weighing⁵
• Independence is prerequisite: multiple subagents investigating simultaneously works best when research paths don't depend on each other⁴. Domains requiring shared context or many dependencies aren't a good fit today³—with dependency, return to chaining's shape
• This lesson's decomposition is all predefined (you wrote the array). Subtasks not predefined but determined by orchestrator based on specific input is the key difference between orchestrator-workers and parallelization¹, also next lesson's topic

>> Lesson 4: Orchestrator-Workers: Making Decomposition Itself Dynamic