The orchestrator-worker multi-agent pattern splits a complex task into a planner that delegates to specialized agents and synthesizes results. It is the most reliable way to keep LLM workflows observable and bounded when you need more than one model call to solve a problem. Teams adopt it because a single prompt cannot both reason about structure and execute isolated transforms without leaking context.
1. Define the decomposition boundary
Start by writing the parent task as a single prompt and manually splitting it into 2–5 subtasks. If you cannot describe each subtask in one sentence with a clear input and output, the orchestrator-worker multi-agent pattern is not ready to be coded. A good test: hand the subtask to a junior engineer and see if they can write a function signature for it.
Map each subtask to a worker role: retrieval, transformation, validation, or generation. Keep the worker count low. Three workers solve most document-processing pipelines; more than six introduces coordination overhead that outweighs model cost. For a contract-review system, the split might be extract_clauses, flag_risks, draft_summary. Each consumes the prior’s structured output, not the raw PDF.
2. Lock worker I/O with schemas
Workers must accept and return structured data, not free text. Define a JSON schema for each worker before writing any agent logic. The orchestrator serializes these contracts; if a worker returns malformed JSON, the orchestrator retries or routes to a fallback, not the user.
{
"worker": "extract_clauses",
"input": {
"document": "string",
"clause_types": ["string"]
},
"output": {
"clauses": [
{"type": "string", "text": "string", "page": "int"}
]
}
}In Python, enforce it with pydantic and a small registry:
from pydantic import BaseModel, Field
from typing import Callable
class ExtractInput(BaseModel):
document: str
clause_types: list[str]
class ExtractOutput(BaseModel):
clauses: list[dict] = Field(default_factory=list)
WORKERS: dict[str, tuple[Callable, type, type]] = {
"extract_clauses": (call_extract, ExtractInput, ExtractOutput),
}Schema versioning belongs in the worker name or a version field. Bump it when fields change.
3. Build the orchestrator planning call
The orchestrator makes one LLM call to produce a plan: which workers to invoke and in what order. Use a strict system prompt and temperature 0. The orchestrator-worker multi-agent pattern lives or dies on this step being deterministic enough to replay.
PLANNER_PROMPT = """You are an orchestrator. Given a task, return JSON:
{"steps": [{"id": int, "worker": str, "depends_on": [int], "input_ref": str}]}
Registered workers: extract_clauses, flag_risks, draft_summary.
Reference earlier step ids in input_ref."""
def plan(task: str) -> list[dict]:
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "system", "content": PLANNER_PROMPT},
{"role": "user", "content": task}],
response_format={"type": "json_object"},
temperature=0
)
data = json.loads(resp.choices[0].message.content)
return data["steps"]Log the raw plan with a task ID. A plan like [{"id":0,"worker":"extract_clauses","depends_on":[],"input_ref":"task"},{"id":1,"worker":"flag_risks","depends_on":[0],"input_ref":0}] is replayable in tests.
4. Dispatch workers concurrently with limits
Workers with no dependencies run in parallel. Use asyncio to cap concurrency with a semaphore and enforce per-worker timeouts. Independent steps should never block each other.
import asyncio
async def run_worker(step, context, sem):
async with sem:
try:
return await asyncio.wait_for(
call_worker(step["worker"], context[step["input_ref"]]),
timeout=20.0
)
except asyncio.TimeoutError:
return {"error": "timeout", "worker": step["worker"]}
async def execute_plan(steps, initial_input):
context = {"task": initial_input}
sem = asyncio.Semaphore(4)
for i, step in enumerate(steps):
deps = [asyncio.create_task(run_worker(steps[d], context, sem))
for d in step.get("depends_on", [])]
if deps:
await asyncio.gather(*deps)
context[i] = await run_worker(step, context, sem)
return contextIf you front your agents with n4n.ai, the gateway’s automatic fallback when a provider is rate-limited keeps worker calls from stalling the orchestrator loop. That removes one class of retry code you would otherwise write.
5. Synthesize with a validation gate
Never return raw worker output to the user. Run a final merge worker that checks invariants and validates schemas. The orchestrator-worker multi-agent pattern produces many intermediate artifacts; the gate is where they become a coherent response.
def synthesize(context, steps):
merged = {}
for i, step in enumerate(steps):
out = context.get(i, {})
if "error" in out:
continue
merged[step["worker"]] = out
# validate required workers succeeded
if "extract_clauses" not in merged or "flag_risks" not in merged:
return {"partial": True, "data": merged}
resp = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "system", "content": "Combine worker outputs into final answer."},
{"role": "user", "content": json.dumps(merged)}]
)
return {"partial": False, "answer": resp.choices[0].message.content}Degrade explicitly. A partial flag lets the caller decide whether to retry or surface a warning.
6. Meter and observe every hop
Per-token usage metering is not optional. Attach a correlation ID to each orchestrator and worker call. Store prompt tokens, completion tokens, and latency per step.
@dataclass
class Span:
worker: str
prompt_tokens: int
completion_tokens: int
ms: float
spans = []
def record(worker, usage, elapsed):
spans.append(Span(worker, usage.prompt_tokens, usage.completion_tokens, elapsed))Aggregate these into a trace. The orchestrator-worker multi-agent pattern generates many small calls; without metering you cannot tell if the decomposition saved money or burned it on serialization round-trips.
7. Test the orchestrator offline
Record planner outputs and worker responses in fixture files. Replay them in CI to catch schema drift and logic regressions.
def test_plan_replay():
steps = plan("Review NDA for liability caps")
assert steps[0]["worker"] == "extract_clauses"
# mock call_worker to return fixturesTreat the planner prompt as code. Any change to worker registry must update the prompt and the test.
Common pitfalls
Over-decomposition. Engineers split tasks into trivial workers that each call an LLM. The planning overhead and serialization loss exceed the benefit. Combine adjacent transforms.
Schema drift. Changing a worker output without versioning breaks the orchestrator silently. Pin schema versions in the planner prompt and in the registry.
Context amplification. Passing full document text between workers multiplies tokens. Pass references or extracted slices instead.
Synchronous blocking. A naive for loop over workers serializes latency. Always use async dispatch for independent steps.
Ignoring partial failure. A single worker timeout should not collapse the entire job. The synthesis gate must handle missing keys.
Tradeoffs versus alternatives
The orchestrator-worker multi-agent pattern adds a planning call and serialization layer. Compared to a single agent with tool calls, it gives stricter isolation: a bad worker cannot corrupt the global prompt. Compared to a flat multi-agent debate, it is cheaper and easier to debug because each hop is typed.
The cost is latency. The planner call adds a round-trip, and structured I/O strips nuance from free-form reasoning. For creative generation, a single agent with few tools often produces better output. The orchestrator-worker multi-agent pattern wins when subtasks are independently verifiable and you care about cost control and audit trails.
When to skip it
If your task is a single retrieval-augmented answer, one model call with a search tool is enough. If you need open-ended exploration, a tree search or agent loop fits better. Use this pattern when you can draw a box diagram of workers with arrows and trust the arrows.
Production checklist
- Worker schemas validated in CI
- Orchestrator plan logged with seed and task ID
- Timeouts and semaphore on every worker
- Fallback model mapped per worker
- Token spans exported to metrics
- Replay tests cover planner and synthesis
Ship the pattern only after the checklist is green.