Glossary
74 terms from “From Loops to Graphs: Orchestration Engineering for Agent Systems.” Hover the first occurrence in the lesson for its definition.
| Term | Definition | Source |
|---|---|---|
| workflow | A system where LLMs and tools are orchestrated through predefined code paths—steps are hardcoded, the model is only autonomous within each individual step. It's one half of an architectural distinction; the five patterns this course teaches all belong to this end. | Building Effective AI Agents — Anthropic Engineering |
| agent | A system where LLMs dynamically direct their own processes and maintain control over how they accomplish tasks. Implementation is often straightforward—typically just LLMs using tools based on environmental feedback in a loop. | Building Effective AI Agents — Anthropic Engineering |
| chaining | Prompt chaining: decomposing a task into a sequence of steps where each LLM call processes the previous one's output, with programmatic checks insertable between steps. Fits when tasks can be cleanly decomposed into fixed subtasks. | Building Effective AI Agents — Anthropic Engineering |
| routing | Classifying an input first, then dispatching to specialized followup tasks, trading for separation of concerns and more specialized prompts. Prerequisite: categories are distinct and classification itself can be handled accurately. | Building Effective AI Agents — Anthropic Engineering |
| parallelization | A workflow where LLMs handle multiple subtasks simultaneously and outputs are aggregated programmatically, with two variations: sectioning and voting. | Building Effective AI Agents — Anthropic Engineering |
| sectioning | Breaking a task into mutually independent subtasks and running them in parallel—like reviewing 12 documents one each. | Building Effective AI Agents — Anthropic Engineering |
| voting | Running the same task multiple times to get diverse outputs, then aggregating via code. | Building Effective AI Agents — Anthropic Engineering |
| orchestrator-workers | A workflow where a central LLM dynamically breaks down tasks, delegates to worker LLMs, and synthesizes results. The key is that subtasks aren't predefined—the orchestrator determines them based on specific input. | Building Effective AI Agents — Anthropic Engineering |
| evaluator-optimizer | One LLM generates a response while another provides evaluation and feedback in a loop, repeating until it passes or stops making progress. Fits when evaluation criteria are clear and iterative refinement provides measurable value. | Building Effective AI Agents — Anthropic Engineering |
| gate | A programmatic check (not another model call) in a chain or loop that uses code to make deterministic checks, same input forever the same conclusion, used to block unqualified intermediate results. | Building Effective AI Agents — Anthropic Engineering |
| harness loop | The while loop driven by stop_reason: send message, check stop reason, if tool_use execute tools and send back, otherwise return. Every 'node' in this course internally runs this loop. | Building Effective AI Agents — Anthropic Engineering |
| stop_reason | The field in each model response stating 'why this turn stopped,' which the harness loop relies on to decide whether to continue calling tools or wrap up. | Building Effective AI Agents — Anthropic Engineering |
| maximum rounds | The iteration ceiling set for a loop, a fuse—guaranteeing code stops under any circumstance. It maintains control, doesn't judge quality. | Building Effective AI Agents — Anthropic Engineering |
| fuse | This course's metaphor for 'maximum rounds' class fallback stopping conditions: shouldn't trigger normally, triggering means other stopping mechanisms didn't catch it. | Building Effective AI Agents — Anthropic Engineering |
| deterministic verifier | A checker that can judge right or wrong via code: if code can judge it, don't spend money asking the model. This course moved it from 'terminal state' to 'between links in the chain.' | Building Effective AI Agents — Anthropic Engineering |
| layered scoring | Use code to judge what code can judge first, only hand to the model what code can't. This course copied this discipline straight into the loop's node ordering (gate before judge). | Building Effective AI Agents — Anthropic Engineering |
| demonstrably | Only consider adding complexity when it demonstrably improves outcomes—every added node or pattern layer must pass this gate. | Building Effective AI Agents — Anthropic Engineering |
| who holds the plan | The axis for judging which end of the determinism spectrum a system falls on: is the plan in code's hands (workflow) or in the model's hands (agent)? | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| the workflow script itself holds the loop, branching, and intermediate results | The sentence from primary materials describing workflow, also the sole primary anchor for this course's 'graph' metaphor: the plan is in the script, the model's context only holds the final answer. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| incrementally tracking | The runtime incrementally tracks each agent's result as the run progresses. This is what makes a run resumable within the same session; cross-process resumption still requires persisting state to disk yourself. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| check-fix-recheck | The review loop's product form: run a checker, fix what failed, repeat until it passes or stops making progress. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| no further progress | A stopping mechanism smarter than maximum rounds: watches whether this round beat the last round (score didn't rise, count of failing items didn't decrease), if it didn't get better then cut losses. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| adversarial cross-review | Have independent agents adversarially review each other's findings before reporting. It's a one-time cross-review before reporting, no back edge, no iteration—lumping it into the review loop is this lesson's categorization, not the original text describing the same topology. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| Composing Patterns into a Graph | This course's own engineering metaphor, not official terminology: a visualization system used to clarify relationships when combining the five patterns. Its sole primary anchor is 'the workflow script itself holds the loop, branching, and intermediate results.' | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| node | A point in this course's visualization: internally either a complete runAgent loop or a piece of pure code. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| back edge | The line in a review loop that sends output back to the generation node. The first four patterns are all straight lines, the review loop is the first with a back edge. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| fork point | A point in this course's visualization for routing where 'walk different edges based on judgment result'—corresponds to branching in code. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| 16 concurrent agents | Claude Code workflow runtime's concurrency ceiling, fewer when CPU is limited; 1,000 agents total ceiling per run. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| 90.2% | Multi-agent system's improvement over single-agent, only holds in complete context: their internal research eval, Opus 4 lead + Sonnet 4 subagents, excelling especially for breadth-first queries. | How we built our multi-agent research system — Anthropic Engineering |
| 4× | By their own data, agents consume about 4× the tokens of chat interactions. | How we built our multi-agent research system — Anthropic Engineering |
| 15× | By their own data, multi-agent systems consume about 15× the tokens of chat interactions; for economic viability, task value must be high enough to justify this improvement. | How we built our multi-agent research system — Anthropic Engineering |
| 90% | After introducing two-level parallelism (lead agent spins up 3-5 subagents in parallel, subagents use 3+ tools in parallel), maximum percentage research time was cut for complex queries. It's a latency number, not quality. | How we built our multi-agent research system — Anthropic Engineering |
| delegation prompts | The task description the orchestrator hands each worker. Delegation prompts must be self-contained with all four elements complete, otherwise workers will duplicate work, leave gaps, or fail to find necessary information. | How we built our multi-agent research system — Anthropic Engineering |
| four elements | The four sections a qualified dispatch must write fully: objective, output format, guidance on tools and sources, clear task boundaries. | How we built our multi-agent research system — Anthropic Engineering |
| Quota rules | Scaling calibration written into the orchestrator's prompt: specify subagent count and per-subagent tool call ceiling by task complexity, because agents aren't good at judging appropriate effort themselves. | How we built our multi-agent research system — Anthropic Engineering |
| scaling rules | Example scaling rules they embedded in prompts: simple fact-finding 1 agent 3-10 calls; direct comparisons 2-4 subagents 10-15 calls each; complex research 10+ subagents with clear division. | How we built our multi-agent research system — Anthropic Engineering |
| synchronous execution | The execution mode where the orchestrator waits for all workers to finish before proceeding. Bottleneck: one step stalls the whole batch waiting, results all return to the orchestrator. | How we built our multi-agent research system — Anthropic Engineering |
| pass references not payloads | Workers store outputs in external systems, passing only lightweight references (file path + one-line summary) back to the orchestrator, not stuffing detailed results back into the main conversation. | How we built our multi-agent research system — Anthropic Engineering |
| artifact systems | Having specialized agents create outputs as independently persisting things (files, external records), rather than relaying everything through the lead agent. Pass references not payloads relies on this. | How we built our multi-agent research system — Anthropic Engineering |
| lightweight references | The short record passed back to the orchestrator: id, file path, one-line summary. Whoever wants the full text reads via the path. | How we built our multi-agent research system — Anthropic Engineering |
| emergent behavior | In multi-agent systems, overall behavior that can't be predicted by looking step-by-step, produced by each subagent's small decisions stacking; tiny changes amplify along the chain. | How we built our multi-agent research system — Anthropic Engineering |
| separation of concerns | Each subagent carries different tools, prompts, exploration trajectories, naturally non-polluting, also reduces path dependency. This is what fan-out buys besides speed. | How we built our multi-agent research system — Anthropic Engineering |
| path dependency | In one loop, later steps' judgments get biased by earlier steps' wording. Multiple independent trajectories don't share the same bias, hence reducing path dependency. | How we built our multi-agent research system — Anthropic Engineering |
| shared context | One scenario multi-agent isn't good at today: work requiring all agents to see the same context or with many inter-agent dependencies (like most coding tasks), truly parallelizable portions fewer than research. | How we built our multi-agent research system — Anthropic Engineering |
| deterministic safeguards | The deterministic mechanisms wrapping non-deterministic agents: retry logic, regular checkpoints. The graph's skeleton is deterministic, node interiors are non-deterministic. | How we built our multi-agent research system — Anthropic Engineering |
| deterministic systems | Systems that produce the same output every time given the same input. Code-written gates, verifiers, aggregation all belong to this category. | Writing effective tools for agents — with agents — Anthropic Engineering |
| non-deterministic | The property of potentially giving different responses even with the same starting conditions—agents are this type. | Writing effective tools for agents — with agents — Anthropic Engineering |
| one loop per evaluation task | Official recommendation for running evaluations: call LLM API directly, use simple agentic while loops, one loop per evaluation task. This course borrows it to illustrate 'a chain stage is a complete loop.' | Writing effective tools for agents — with agents — Anthropic Engineering |
| subagent | An agent dispatched from the main conversation, works in its own independent context window, only hands summary back. Used to keep exploration and implementation out of the main conversation, preserving context. | Create custom subagents — Claude Code official documentation |
| main conversation | The orchestrator's main-line context. When subagents finish, results return here, so returns must be light or protection becomes burden. | Create custom subagents — Claude Code official documentation |
| 20 subagents | Claude Code's default concurrency ceiling: when 20 subagents are running in a session, spawning another fails and explicitly tells the model not to retry. | Create custom subagents — Claude Code official documentation |
| subagents sequentially | Claude Code's suggestion for multi-step workflows: have Claude use subagents sequentially, each finishes and hands results back, then pass relevant context to the next. This is chaining's product form. | Create custom subagents — Claude Code official documentation |
| Specialization | Routing work to agents with domain-specific system prompts and tools (security agent, documentation agent), rather than stuffing all capabilities into one agent. Routing's name in current first-party vocabulary. | Multiagent orchestration — Claude API documentation (Managed Agents) |
| Escalation | Consulting a more capable agent or model for a subset of complex subtasks. Note it's 'consult' (coordinator still holds control), not complete handoff. | Multiagent orchestration — Claude API documentation (Managed Agents) |
| Parallelization | In current Claude platform multi-agent orchestration docs: fan out independent subtasks in parallel (search multiple sources, analyze separate files), coordinator synthesizes results. | Multiagent orchestration — Claude API documentation (Managed Agents) |
| 25 concurrent threads | Managed Agents' supported concurrency ceiling; coordinator can call multiple copies of the same agent, each opening a thread. | Multiagent orchestration — Claude API documentation (Managed Agents) |
| concurrency pool | Open a fixed N lanes, each finishes its current item then grabs the next from a shared cursor, always N in flight. Has less bucket effect than batching. This course's script calls it runPool. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| concurrency ceiling | The hard cap on simultaneous in-flight tasks. Three real products (Claude Code 20, workflow runtime 16, Managed Agents 25) all set one—unbounded fan-out is an accident not an optimization. | Create custom subagents — Claude Code official documentation |
| batching | A rate-limiting approach: N per batch, batches serial. Works but has bucket effect: each batch waits for that batch's slowest to finish before starting the next. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| Promise.all | The pattern for running a group of async tasks concurrently, returning when all complete. Temperament: any one path rejects, the whole rejects—even if the rest finished, you can't get their results. | Building Effective AI Agents — Anthropic Engineering |
| fan out | Spreading one piece of work to multiple agents/nodes to run simultaneously. Buys speed, independent perspectives, parallel context capacity; cost is results return to orchestrator, real products all set ceilings. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| merge | After fan-out results come back, code does filtering, aggregation, judgment. This course has it only read references back from files, not stuff full text into context. | Building Effective AI Agents — Anthropic Engineering |
| payload | The detailed content of worker output itself. Payload backflow at merge explodes context, so pass references not payloads. | How we built our multi-agent research system — Anthropic Engineering |
| classifier | The first step in routing: judge which category input belongs to, output tightened to one word. Can be a model or a traditional classification model/algorithm. | Building Effective AI Agents — Anthropic Engineering |
| fallback | The branch walked when classification doesn't fit the legal label table (like falling to 'other'). Model occasionally returns a whole sentence instead of one word, fallback contains this non-determinism in one line. | Building Effective AI Agents — Anthropic Engineering |
| determinism spectrum | This course's own phrasing, not official terminology: treating that 'who holds the plan' axis from workflow to agent as a continuous range from deterministic to autonomous. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| controlled crash | Course 9 in this series' practice: deliberately stop the process at a specified point to verify state actually fell to disk. This course's STOP_AFTER is its simplified version. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| spin | The state where a loop does the same thing for multiple consecutive rounds without any progress. Course 7 in this series' Valve 3 (spin detection) is for blocking this. | Building Effective AI Agents — Anthropic Engineering |
| atomic write | The disk persistence method of writing .tmp first then rename to replace: killed at any moment, disk has either the previous complete state or the new complete state, never half a JSON. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| run-state.json | This course's script's execution tracking file (persisted once after each node completes, again after each judgment in review), atomic write guarantees it's readable at any moment. It records execution progress, not 'the graph's state variables.' | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| run.jsonl | This course's script's structured log: one JSON event per line, carrying ts and run_id, only records id/category/filename/counts, not reply body. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| stub client | A test stand-in that uses a fixed response queue to impersonate the real API: client.messages.create() returns preset responses in order, letting the whole orchestration run zero-dependency and reproducibly. | Writing effective tools for agents — with agents — Anthropic Engineering |
| STOP_AFTER | This course's script's controlled crash switch: set to merge stops before review, exit code 2, used to verify fan-out/merge state actually persisted. Only recognizes merge as one value. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |
| needs_human | The outcome marker when the loop can't produce qualified output: exit code 1 tickets stop here, handed to human. | Orchestrate subagents at scale with dynamic workflows — Claude Code official documentation |