Most agent loops degrade not because the model is weak, but because the prompt leaves too much implicit. Adding few-shot examples agent prompts can compress hours of trial-and-error into a single inference call—if you place them where the model actually attends. This guide walks through when examples earn their token cost and when they just add noise.
Why few-shot works differently for agents
In a single-turn completion, few-shot examples teach surface format. In an agent, the model must map observations to actions under a changing environment. Few-shot examples agent prompts should demonstrate reasoning about state transitions, not just output shape.
Agents consume prompts that mix system instructions, tool schemas, and rolling history. The example must survive that context window without getting buried under tool output or user chatter.
Step 1: Identify the decision boundary you need to fix
Don’t add examples because a blog said so. Profile your agent’s failure modes: does it call the wrong tool, misorder steps, or hallucinate parameters? Few-shot helps when the error is consistent and the correct behavior is demonstrable in a short trajectory.
Write down the exact trigger:
Failure: When user asks "summarize and file", agent calls summarize then stops.
Expected: call summarize, then call file_doc with returned id.That is a boundary you can show in one example. If the failure is sporadic or rooted in retrieval quality, examples won’t fix it.
Step 2: Collect real traces, not synthetic ideals
Pull actual agent runs from logs. Synthetic examples built from docs tend to be too clean; the model already knows the happy path. The value is in messy inputs: truncated tool output, ambiguous user intent, partial errors.
A minimal example pair from a trace:
{
"messages": [
{"role": "user", "content": "summarize the ticket and file it"},
{"role": "assistant", "content": "{\"tool\": \"summarize\", \"args\": {\"ticket\": 4821}}"},
{"role": "tool", "content": "{\"summary\": \"login fails on Safari\"}"},
{"role": "assistant", "content": "{\"tool\": \"file_doc\", \"args\": {\"summary\": \"login fails on Safari\"}}"}
]
}Use this as a template for your few-shot block. Strip credentials and PII, but keep the awkward phrasing—that is the signal.
Step 3: Format examples as agent-observable state
The model attends to structure it recognizes from its own generation. If your agent emits JSON tool calls, the example must use that exact schema. If it interleaves thought tags, include them.
A common mistake is showing only the final answer. For agents, show the observation-action chain:
def format_example(trace):
lines = []
for m in trace["messages"]:
if m["role"] == "user":
lines.append(f"User: {m['content']}")
elif m["role"] == "assistant":
lines.append(f"Agent: {m['content']}")
elif m["role"] == "tool":
lines.append(f"Tool: {m['content']}")
return "\n".join(lines)This keeps the example inside the same distribution as live inference. Mismatched formatting teaches the model to diverge from your parser.
Step 4: Place examples near the relevant instruction
Prompt order matters. Put the few-shot examples agent prompts directly above the instruction that governs the behavior, not in a distant appendix. For tool-selection fixes, insert before the tool list:
# Examples of correct tool chaining
User: summarize and file
Agent: {"tool":"summarize","args":{"id":1}}
Tool: {"summary":"ok"}
Agent: {"tool":"file_doc","args":{"summary":"ok"}}
# Available tools
- summarize: returns summary for ticket id
- file_doc: stores summary to doc storeIf the system prompt is cached, prefix examples at the start to benefit from provider cache hits. Gateways that forward cache-control hints—n4n.ai does this on its OpenAI-compatible endpoint—let you mark the static few-shot prefix so repeated agent calls pay less per token.
Step 5: Cap the count and rotate
Three to five examples is enough for most boundaries. Beyond that, you risk overfitting to the demonstrated style and bloating context. Rotate examples based on recent failure clusters:
def select_examples(failures, pool, k=3):
# pick pool items whose trigger matches current failure signature
matched = [e for e in pool if e["trigger"] in failures]
return matched[:k]Store the pool in version control. Treat it like test fixtures, not prompt decoration.
Pitfalls: token bloat and distribution shift
Every example costs input tokens on every call. At 10 examples of 400 tokens each, that’s 4k tokens of overhead per step in a 20-step loop—wasteful if the behavior is already correct 95% of the time.
Distribution shift hits when your examples reflect last month’s tool schema. An agent prompted with outdated parameter names will confidently repeat them. Audit examples whenever you change tools.
Another trap: examples that show recovery from errors can teach the model to induce those errors. If you demonstrate “call fails, then retry”, the model may learn to fail first to match the pattern.
Tradeoffs: static vs dynamic example selection
Static few-shot (baked into system prompt) is simple and cache-friendly. Dynamic selection (retrieve relevant examples per query) reduces token waste but adds a retrieval step and latency.
// pseudo retrieval
const ex = await embed(query);
const top = await vectorStore.search(ex, 2);
prompt = [system, ...top, instruction];Use dynamic only when you have many distinct failure modes (>20) and a clear embedding signal. Otherwise static wins on reliability.
When to skip few-shot entirely
If the agent fails due to missing tools or bad retrieval, examples won’t help. If the model already follows the instruction zero-shot in eval, adding examples just costs tokens. Run an ablation: remove the block and measure task success. Keep it only if the delta is positive and reproducible.
A minimal end-to-end template
Below is a compact system prompt section using few-shot examples agent prompts for a filing agent:
{
"system": "You are a ticketing agent. Chain tools when asked.",
"few_shot": [
{
"user": "summarize and file",
"agent_1": {"tool": "summarize", "args": {"id": 1}},
"tool": {"summary": "ok"},
"agent_2": {"tool": "file_doc", "args": {"summary": "ok"}}
}
],
"tools": [{"name": "summarize"}, {"name": "file_doc"}]
}Serialize this into the chat format your gateway expects. Keep the few_shot block immutable across calls to maximize cache reuse.
Closing checklist
- One decision boundary per example set.
- Real traces, not docs.
- Exact schema match.
- Placed adjacent to instruction.
- Capped at 5, rotated by failure signature.
- Ablated before shipping.
Few-shot examples agent prompts are a scalpel, not a blanket. Use them where the model’s prior is wrong and the fix is short. Everywhere else, they’re tax.