How to structure multi-step instructions for AI agents

Writing effective multi-step instructions for AI agents requires treating the prompt as a program with typed transitions, not a paragraph of wishes. The difference shows up the first time your agent silently skips a validation step or loops on a retry instead of escalating.

1. Encode steps as a numbered contract

Agents execute better when the instruction set looks like a state machine, not a story. Each step needs an identifier, a defined input source, an output target, and an explicit failure action. Vague sequences like “first research, then write, then check” leave the model to infer control flow, and it will infer wrong under load.

Write the contract as data. JSON works because you can validate it and feed it back to the model as constrained context:

{
  "steps": [
    {"id": "fetch", "input": "user_query", "output": "raw_docs", "on_fail": "abort"},
    {"id": "summarize", "input": "raw_docs", "output": "brief", "on_fail": "retry:2"},
    {"id": "cite", "input": "brief", "output": "final", "on_fail": "ask_clarification"}
  ]
}

The pitfall here is overloading the step list with reasoning hints. Keep policy separate from tactic. If you need to tell the model how to summarize, put that in a step-specific directive block, not in the master sequence.

Tradeoff: a rigid contract reduces emergent flexibility. For open-ended creative tasks, a loose outline beats a strict schema. For automation workflows, strict wins every time.

2. Split system instruction from step policy from live context

Three concerns rot together if you paste them into one giant prompt: who the agent is, what order it must follow, and what data it currently holds. Separate them at build time even if they get concatenated at send time.

SYSTEM = "You are a compliance research agent. Never invent regulations."
STEP_POLICY = ["fetch", "summarize", "cite"]

def build_prompt(ctx: dict) -> str:
    return f"{SYSTEM}\nActive steps: {STEP_POLICY}\nCurrent data: {ctx}"

This separation lets you swap the step policy without retesting the persona. It also makes diffs readable when something breaks. A common bug is binding a transient user variable into the system prompt, which then leaks across sessions.

The cost is minor boilerplate and one extra string format. The gain is debuggability: you can log which layer produced a bad output.

3. Define strict schemas for inter-step data

Free-form text between steps is where agents drift. If step A outputs a paragraph and step B must extract URLs, you have built a silent parser. Use a schema.

from pydantic import BaseModel, Field

class FetchResult(BaseModel):
    docs: list[str] = Field(description="Raw extracted text blocks")
    source_errors: list[str] = Field(default_factory=list)

class Brief(BaseModel):
    summary: str
    confidence: float

Now the executor can reject a step that returns malformed JSON. The model self-reports errors inconsistently; a validator does not. Pitfall: over-constraining fields the model can’t reliably produce (e.g., exact timestamps). Allow optional fields and degrade gracefully.

Tradeoff: schema validation adds latency and sometimes forces the model to wrap output in code fences. Use response_format where the provider supports it, but keep a fallback parser.

4. Choose a planner-executor or linear sequencer

Two patterns dominate production agents. The linear sequencer runs a fixed step list. The planner-executor asks a model to generate a step list per query, then executes it.

def run_linear(query):
    ctx = {"user_query": query}
    for step in STEP_POLICY:
        ctx = execute_step(step, ctx)
    return ctx["final"]

def run_planner(query):
    plan = planner_model.generate_steps(query)  # returns ordered step ids
    ctx = {"user_query": query}
    for step in plan:
        ctx = execute_step(step, ctx)
    return ctx["final"]

Linear is cheaper and predictable. Planner adapts to novel tasks but can emit invalid plans. I ship linear first, then add a planner only for the long tail of queries that fail.

Pitfall: letting the planner re-write the system instruction. Constrain its output to step IDs from your contract.

5. Make failure paths explicit with bounded retries

Unhandled failures become retries without limit, which burns tokens and hangs the user. Your multi-step instructions for AI agents must specify retry caps and a fallback terminal state.

{
  "id": "summarize",
  "on_fail": "retry:3",
  "fallback_step": "ask_clarification",
  "timeout_ms": 20000
}

Implement the cap in code, not just in the prompt. The model will not reliably count to three. When a provider is degraded, an inference gateway with automatic fallback can mask the outage, but your loop still needs a hard ceiling.

Tradeoff: aggressive fallback improves robustness but can surface partial results that look finished. Always tag fallback outputs with a flag the UI can show.

6. Route model capacity per step

Not every step needs a frontier model. Extraction and formatting can run on a small model; synthesis and ambiguity resolution need the large one. If you use a gateway that honors client routing directives, pin the model per step in the request. n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models and honors those directives, so you can send model: "step-extract" for cheap steps and reserve the heavy model for step-cite.

def execute_step(step, ctx):
    model = "step-extract" if step in ("fetch",) else "step-synthesize"
    return client.chat.completions.create(
        model=model,
        messages=[{"role": "system", "content": STEP_TEMPLATES[step]}]
    )

This keeps per-token cost sane. The pitfall is routing a step that needs judgment to a model that hallucinates structure. Test each step in isolation before composing.

7. Test with replay and golden transcripts

Prompt changes are code changes. Capture a set of input contexts and expected intermediate outputs. Re-run the sequence and diff.

pytest tests/agent_replay.py --snapshot-update

Store the raw model responses, not just the parsed objects. When a step regresses, you can see if the model ignored a schema or the validator broke. A common mistake is testing only the happy path; inject a source_errors case and a timeout case.

Tradeoff: snapshot tests are brittle when providers update models. Pin model versions in CI for the parts you validate.

8. Pitfalls and tradeoffs summary

• Hidden state: If the agent stores anything outside the explicit context object, you can’t replay it. Force all state through the schema.
• Over-specification: Too many constraints make the model refuse valid inputs. Leave escape hatches like notes fields.
• Cost vs control: Strict multi-step instructions for AI agents cost more in engineering time but cut runtime waste from loops.
• Provider coupling: Writing fallback logic against one provider’s error codes breaks on swap. Use an OpenAI-compatible interface and map errors generically.

The teams that ship reliable agents treat instructions as compiled artifacts: versioned, validated, and measured per step. Start with a linear contract, add schemas, then earn the right to plan.