Getting an LLM to invoke your functions correctly is less about clever phrasing and more about constraint design. To prompt agents for tool use reliably, you must treat the model as a junior engineer with a strict spec, not a mind-reader. The following steps take you from loose instructions to a validated orchestration loop you can ship.
Step 1: Define machine-checkable tool schemas
A tool description like “search the database” invites malformed calls. Write a JSON Schema that pins every parameter type, range, and required field. The model sees the schema in the tools array; your code uses the same schema to reject bad input before it touches production systems.
{
"type": "function",
"function": {
"name": "query_orders",
"description": "Fetch order rows by status and date range.",
"parameters": {
"type": "object",
"properties": {
"status": { "type": "string", "enum": ["open", "shipped", "cancelled"] },
"start_date": { "type": "string", "format": "date" },
"end_date": { "type": "string", "format": "date" }
},
"required": ["status", "start_date"]
}
}
}Avoid default values in the schema; they let the model skip arguments silently. If a parameter is optional, say so in the description and handle null in code. Naming matters: query_orders beats get_data. Verb-noun pairs map cleanly to intent.
Do not let two tools overlap in purpose. If you expose get_user and fetch_customer, the model will guess. Keep a one-to-one mapping between user intent and tool name. For nested parameters, define explicit sub-schemas rather than accepting free-form objects—validation is only as strong as the deepest leaf.
Step 2: Write a system prompt that constrains calling behavior
The schema answers what the tool needs; the system prompt answers when to call and what to do on failure. Be prescriptive. Ambiguity is the enemy of reliable agents.
You are an order-support agent. Rules:
1. Call query_orders when the user references orders, shipments, or cancellations.
2. If start_date is missing, ask for it in one sentence. Do not invent dates.
3. After a tool returns, state the row count and the top result.
4. If the tool errors, retry once with tightened args. If it fails again, say "unable to retrieve".
5. Never call a tool and answer from memory in the same turn.Use “must” and “never”, not “you can” or “try to”. This style—numbered directives—outperforms prose because the model can index rules easily. To prompt agents for tool use at scale, version this prompt like a config file and diff changes when call accuracy drifts.
Set temperature low (0.2 or below) for the orchestration model. High randomness destroys schema adherence. The system prompt is not the place for personality; keep it terse and operational.
Step 3: Seed context with explicit invocation examples
Few-shot turns teach structure faster than rules. Include a happy path and a refusal path.
[
{"role": "user", "content": "Show me shipped orders since May 1"},
{"role": "assistant", "tool_calls": [
{"id": "c1", "type": "function", "function": {"name": "query_orders", "arguments": "{\"status\":\"shipped\",\"start_date\":\"2024-05-01\"}"}}
]},
{"role": "tool", "tool_call_id": "c1", "content": "{\"rows\":12,\"sample\":{\"id\":881,\"total\":42.00}}"},
{"role": "assistant", "content": "Found 12 shipped orders. First: #881, $42.00."},
{"role": "user", "content": "orders from last week"},
{"role": "assistant", "content": "What status? open, shipped, or cancelled?"}
]The second example shows the missing-argument refusal from rule 2. Models pattern-match the shape of the assistant message, so keep your few-shot tool_calls byte-compatible with the API. Add one negative example where the model should not call a tool—e.g., a pure chit-chat greeting—to suppress over-calling.
Step 4: Implement the orchestration loop with validation
Your service owns the loop. Below is a minimal but complete Python runner using the OpenAI SDK. It validates every call against the schema before execution and safely parses arguments.
import json
import jsonschema
from openai import OpenAI
client = OpenAI() # swap base_url for a gateway if needed
def execute_tool(name, args):
if name == "query_orders":
return {"rows": 0, "sample": None} # stub
raise ValueError("unknown tool")
def run_agent(messages, tools):
while True:
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
tools=tools,
tool_choice="auto"
)
msg = resp.choices[0].message
if not msg.tool_calls:
return msg.content
messages.append(msg)
for call in msg.tool_calls:
fn = call.function
try:
args = json.loads(fn.arguments)
except json.JSONDecodeError:
messages.append({"role": "tool", "tool_call_id": call.id,
"content": "PARSE_ERROR: arguments not valid JSON"})
continue
spec = next(t for t in tools if t["function"]["name"] == fn.name)
try:
jsonschema.validate(args, spec["function"]["parameters"])
except jsonschema.ValidationError as e:
messages.append({"role": "tool", "tool_call_id": call.id,
"content": f"VALIDATION_ERROR: {e.message}"})
continue
try:
result = execute_tool(fn.name, args)
except Exception as e:
messages.append({"role": "tool", "tool_call_id": call.id,
"content": f"TOOL_ERROR: {type(e).__name__}: {e}"})
continue
messages.append({"role": "tool", "tool_call_id": call.id,
"content": json.dumps(result)})If you point client = OpenAI(base_url="https://api.n4n.ai/v1") the same loop gains automatic fallback when a provider is rate-limited, because the gateway fronts 240+ models with one OpenAI-compatible shape. That keeps your retry logic about semantics, not transport.
Step 5: Handle errors and retries with policy
Validation catches type errors; tools can still throw at runtime. Feed the error back as a tool message and let the model self-correct once. Cap the loop at 5 iterations. After that, return a canned “I couldn’t complete that” to stop token bleed.
Idempotency is your responsibility. If execute_tool writes to a database, use the tool_call_id as a dedupe key. The model may resend the same call after a transient error; your code should absorb it.
Step 6: Verify success with contract tests
You cannot ship what you don’t test. Write a unit test that drives run_agent with a stubbed execute_tool and asserts the expected call name and args.
def test_query_orders_called():
tools = [{"type": "function", "function": {
"name": "query_orders",
"parameters": {"type": "object", "properties": {"status": {"type": "string"}}, "required": ["status"]}
}}]
messages = [{"role": "user", "content": "shipped orders"}]
run_agent(messages, tools)
calls = [m for m in messages if m.get("tool_calls")]
assert any("query_orders" in str(c.tool_calls) for c in calls)Run this in CI on every prompt change. For production confidence, replay the last 100 real conversations through the loop in shadow mode and check validation error rate.
How to verify in production
Log each tool call with the schema version and validation outcome. Alert if error rate on a route exceeds 2%. To prompt agents for tool use reliably, you watch the boundary, not the model’s mood.
Step 7: Batch and parallelize only with explicit directives
Default model behavior is sequential. If your task needs parallel calls, say it.
If the user asks for two distinct statuses, emit two tool_calls in one assistant message.Confirm with a test: prompt “open and cancelled orders” should yield two calls, not a question. Without the directive, many models ask follow-ups. Parallel calls reduce latency but require your execute_tool to be thread-safe.
Step 8: Treat prompts as versioned code
Tag your system prompt and few-shot set with a hash. When you update the schema, bump the tag. Keep a changelog of accuracy on a fixed eval set. This discipline is what separates a demo from a system.
Build a small eval corpus of 50 representative requests with known correct tool calls. Run it on every model swap. If you use a gateway that honors client routing directives, pin the model per environment so results stay comparable.
Reliable tool use is an engineering problem. Define contracts, enforce them in code, and measure. The phrase “prompt agents for tool use” really means “build a validator around a language model.”