Designing prompts for multi-agent handoffs

Multi-agent systems split work across specialized models, but the transition between them is where most failures originate. Designing prompts for multi-agent handoffs is less about clever phrasing and more about enforcing a strict contract that preserves intent, constraints, and intermediate state. Treat the handoff as a typed message, not a conversation, or you will burn tokens replaying lost context.

1. Model the handoff as a schema, not a summary

Prose handoffs (“here’s what the user wants, go do it”) lose structure the moment a second agent interprets them. Define a JSON schema that every sender fills and every receiver parses. The schema is the API between your agents.

{
  "handoff_id": "h_8f2c",
  "parent_id": null,
  "from_agent": "planner",
  "to_agent": "sql_executor",
  "task": "Retrieve monthly revenue for acme in 2023",
  "constraints": {
    "max_rows": 100,
    "timeout_ms": 5000,
    "read_only": true
  },
  "context": {
    "schema": "sales(id, account, amount, ts)",
    "prior_attempts": 0
  },
  "success_criteria": "Return JSON with columns month, total",
  "on_failure": "Return error code and abort",
  "override_policy": "receiver_may_abort_if_constraint_violation"
}

Include handoff_id and parent_id from the start. Agents loop; without lineage you cannot tell a retry from a new branch. Keep the schema minimal: only fields the receiver must act on. A 30-field blob invites the planner to hallucinate filler.

2. Write the sender prompt to emit the contract

The planner’s system prompt must forbid free-form text. Instruct it to output only the handoff object, and validate against the schema before forwarding. Use response-format enforcement when the model supports it.

system = """You are a planning agent. Given a user goal, emit a handoff JSON matching schema X.
Never output explanatory text. If the goal is ambiguous, set on_failure to 'clarify'.
Do not invent fields not defined in the schema."""

user = "Get me acme's 2023 revenue by month"

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role":"system","content":system},{"role":"user","content":user}],
    response_format={"type":"json_object"}
)
raw = resp.choices[0].message.content
# strip occasional "```json" wrappers if the model ignores format
if raw.startswith("```"):
    raw = raw.split("```")[1]
handoff = json.loads(raw)

Pitfall: models hedge with “Sure, here is the JSON”. Enforce response_format or post-process. Another pitfall is the sender embedding the full chat history in context. Pass only derived facts.

When routing this call through an OpenAI-compatible gateway such as n4n.ai, you can pin a specific model but still get automatic fallback if that provider is degraded, keeping the handoff pipeline moving without code changes.

3. Receiver prompt reconstructs state from the object

The executor should not re-plan. Its prompt consumes the handoff and maps fields to tool calls. The receiver is stateless aside from the handed object.

exec_system = """You are a SQL executor. You receive a handoff JSON. Use its task, constraints,
and context to build a read-only query. Output only the result JSON defined in success_criteria.
If constraints cannot be met, return the on_failure structure with a code."""
exec_messages = [
    &#123;"role":"system","content":exec_system&#125;,
    &#123;"role":"user","content":json.dumps(handoff)&#125;
]

3.1 Avoid context regurgitation

Do not ask the receiver to “remember the user is a CFO”. That lives in context. If the receiver needs identity, put context.user_role. Prompts that rely on shared memory across agents silently break when you swap the receiver model.

3.2 Map fields to tool schemas

If the receiver uses function calling, translate task and constraints into the tool’s parameters explicitly in the prompt:

The handoff.constraints.max_rows maps to your query's LIMIT.
The handoff.context.schema is the only tables you may reference.

Tradeoff: tight coupling. If the schema changes, both prompts break. Mitigate by rendering both prompts from a single Python dataclass so the field names cannot drift.

4. Encode authority and escalation

Handoffs need a clear chain of command. Who wins if the receiver thinks the task is impossible? Without policy, agents loop: planner sends, executor asks, planner re-sends.

Add override_policy to the schema and prompt the receiver to act on it:

If handoff.constraints.read_only is true and the only way to satisfy task is a write,
immediately return on_failure with code FORBIDDEN. Do not ask the planner.

Valid policies:

receiver_may_abort_if_constraint_violation (default safe)
receiver_must_attempt_and_report
receiver_may_escalate_to_human

Designing prompts for multi-agent handoffs without this field produces polite agents that never fail loudly.

5. Log the exact handoff prompt for replay

Multi-agent bugs are intermittent. Store the rendered sender prompt, the raw model output, and the parsed handoff. Use per-token metering to attribute cost to each handoff step.

# example structured log line
&#123;"ts":1700000000,"agent":"planner","handoff_id":"h_8f2c",
 "prompt_tokens":412,"completion_tokens":88,"payload":&#123;"task":"...","constraints":&#123;...&#125;&#125;&#125;

Replay by feeding the same prompt to a cheaper model to reproduce drift. If you use a gateway that honors client routing directives, you can route replays to a low-cost model without touching orchestration code.

6. Test handoffs with constrained simulations

Do not test the full system with frontier models on every commit. Use a small model for the receiver and a scripted sender fixture.

// pseudo-orchestrator test
const handoff = loadFixture("planner_output.json");
const out = await runAgent("sql_executor", handoff, &#123;model:"mistral-7b-instruct"&#125;);
assert(out.columns.includes("month"));
assert(out.rows.length &lt;= handoff.constraints.max_rows);

Pitfall: small models drop constraints. Add a validation layer that rejects handoffs missing required fields before they reach the receiver.

REQUIRED = ["handoff_id","to_agent","task","success_criteria"]
def validate(h):
    missing = [k for k in REQUIRED if k not in h]
    if missing:
        raise HandoffError(f"missing &#123;missing&#125;")

Run these tests in CI with frozen fixtures so prompt changes surface as diffs.

7. Common pitfalls and tradeoffs

Over-specifying: A schema with 30 fields makes the planner spend tokens filling junk. Keep it minimal.
Under-specifying: No success_criteria means the receiver guesses output shape and the next agent parses garbage.
Bidirectional handoffs without IDs: Use handoff_id and parent_id to trace loops and prevent infinite cycles.
Ignoring cache hints: Forward provider cache-control on long system prompts to cut cost on repeated handoffs.
Hidden state: Putting data in the orchestrator’s memory instead of the handoff breaks when you add a third agent.

Tradeoff: latency vs. robustness

Synchronous handoffs are easy to debug but block the user thread. Async message queues decouple agents but require idempotent receivers and correlation IDs. Choose sync for interactive flows under a few seconds; choose async for long-running research or batch jobs.

Tradeoff: model tier per agent

Planners benefit from stronger reasoning; receivers often need only instruction following. Designing prompts for multi-agent handoffs lets you swap a receiver to a 7B model safely because the contract is explicit. The planner still needs capacity to emit valid JSON.

8. Putting it together

A minimal orchestrator loop with validation and error capture:

def step(agent, handoff):
    prompt = build_prompt(agent, handoff)
    res = call_model(prompt, agent.model)
    parsed = parse_handoff(res)
    validate(parsed)
    return parsed

handoff = step("planner", &#123;"task": user_goal&#125;)
steps = 0
while handoff.to_agent != "human" and steps &lt; 5:
    steps += 1
    try:
        handoff = step(handoff.to_agent, handoff)
    except HandoffError as e:
        log_error(handoff.handoff_id, e)
        break

The loop is boring on purpose. Interesting agents are a symptom of unclear handoffs.

Designing prompts for multi-agent handoffs is fundamentally about making the inter-agent boundary explicit and machine-checkable. Do that and your agents compose; skip it and you get a chatroom with extra steps.