The debate over system prompts vs tool descriptions is mostly solved in practice, but the patterns that work are rarely written down. When you build an agent, the system prompt sets the agent’s operating identity and guardrails, while tool descriptions teach the model what actions exist and how to invoke them. Getting this split wrong is the difference between an agent that reliably follows policy and one that hallucinates API calls or ignores constraints.
The division of labor
What a system prompt actually controls
A system prompt is the only text guaranteed to sit in every completion’s context window before the user speaks. It sets role, objectives, and non-negotiable constraints. In an agent loop, it is the closest thing to a process handbook. It tells the model what kind of entity it is, what success looks like, and which actions are off-limits regardless of tool availability.
But the system prompt is natural language. The model weights it against everything else, including tool schemas and recent user messages. It is a soft contract. If you write “always confirm with user before deleting”, a distracted model can still skip that step when a tool named delete_all looks convenient. The prompt sets intent; it does not enforce mechanics.
What a tool description actually controls
Tool descriptions are structured contracts rendered as JSON Schema plus free text. The model parser uses the schema to validate arguments before the call leaves your infrastructure. The description string informs the decision of whether to call. A well-written description includes the action’s effect, required preconditions, and failure modes.
In the system prompts vs tool descriptions debate, the tool side wins on precision. You can encode an enum, required fields, and types. The model cannot emit a string where an integer is required if your schema is enforced server-side. The description text is where you explain nuance: “This tool bills the customer immediately” changes behavior more than a system note buried under policy.
Why engineers conflate them
The confusion starts because both influence action selection. Early agent code often treats the system prompt as a dumping ground. I have seen prompts with 2,000 tokens of API documentation copied from README files. The developer hoped the model would “just know” how to call the endpoint. It does not. The model uses tool schemas as the primary signal for function calling; long prose in the system prompt gets attenuated.
Example: a refund agent
Take a support agent with two tools: request_approval and issue_refund. The system prompt states: “You must get manager approval before refunding.” The issue_refund description says merely: “Refunds an order.” A new model version, or a model from a different provider, may treat the tool as self-contained and skip the approval step because the schema does not require an approval token. The policy existed only in natural language.
Fix by moving the precondition into the tool:
{
"type": "function",
"function": {
"name": "issue_refund",
"description": "Issue a refund. Side effect: moves money to customer. REQUIRES approval_token from request_approval.",
"parameters": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"amount_cents": {"type": "integer", "minimum": 1},
"approval_token": {"type": "string", "description": "Token from request_approval."}
},
"required": ["order_id", "amount_cents", "approval_token"]
}
}
}Now the constraint is structural. The system prompt keeps the higher-level rule, but the tool makes violation impossible at the API boundary.
Concrete patterns that work
System prompt skeleton
Keep it under 300 tokens. Lead with role, then constraints, then escalation. Avoid parameter docs.
Role: Billing support agent for Acme.
Goal: Resolve user billing issues using provided tools.
Constraints:
- Never refund without approval_token in tool call.
- Never reveal internal error traces.
- If tool fails twice, open ticket.
Escalation: Use open_ticket for anything outside tools.Tool description schema
Every parameter gets a description. Every side effect gets a sentence. If a value is restricted, use schema enums, not prose.
def usage_tool():
return {
"type": "function",
"function": {
"name": "query_usage",
"description": "Read-only fetch of account usage. No side effects.",
"parameters": {
"type": "object",
"properties": {
"account_id": {"type": "string", "description": "Acme UUID."},
"metric": {"type": "string", "enum": ["cpu", "storage", "network"],
"description": "Metric to return."}
},
"required": ["account_id", "metric"]
}
}
}Minimal agent loop
Using the OpenAI client against any OpenAI-compatible endpoint:
from openai import OpenAI
client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="...") # or your gateway
SYS = open("system_prompt.txt").read()
tools = [refund_tool(), usage_tool(), approval_tool()]
messages = [{"role": "system", "content": SYS}]
while True:
u = input("> ")
messages.append({"role": "user", "content": u})
r = client.chat.completions.create(model="gpt-4o-mini", messages=messages, tools=tools)
# handle tool_calls, append results, continueThis loop works identically if the model behind the endpoint changes, provided your tool schemas are valid.
Tradeoffs and failure modes
Overstuffing the system prompt
Symptoms: high token cost, degraded instruction following, broken fallback to smaller models. The system prompt is not a schema. When you embed JSON examples in prose, you waste context that could be spent on conversation. Worse, small models truncate system text; your critical rules vanish.
Under-specifying tools
A description like “Manages orders” yields hallucinations. The model fills parameters from guesswork. Always state side effects. If a tool sends email, say “Sends email immediately”. If it is idempotent, say so. The model chooses tools based on perceived effect; vague text leads to wrong choices.
Dynamic tool injection
In plugin architectures, tools load per request. Then the system prompt must stay generic: “You have tools to manage billing.” Each tool carries its own spec. This decoupling is the correct end state for large agent systems. It also makes the system prompts vs tool descriptions boundary crisp: prompt is constant, tools are variable.
Parameter enforcement beats prose
Whenever you can express a rule as schema, do it. Enums, minimums, patterns, required arrays are machine-checked. The model cannot argue with a missing required field; your server rejects the call. Reserve the system prompt for judgments that cannot be enumerated: tone, prioritization, ethical lines.
Example: instead of “Only query metrics cpu, storage, network” in prompt, use the enum above. The prompt can mention “prefer cpu for performance issues” as guidance.
Model portability and gateways
If you serve agents through a gateway that aggregates many models, the split matters for maintenance. A gateway that honors client routing directives and forwards provider cache-control hints—like n4n.ai—lets you pin one tool schema while the backend shifts from a large model to a smaller one under load. Your system prompt should avoid phrases like “as a GPT-4 class model”. Keep it model-agnostic. Tool descriptions in OpenAI function format are portable across providers that support the spec; the system prompt is portable only if you keep it generic.
Debugging misbehavior
When an agent calls the wrong tool, inspect the description similarity first. If two tools sound alike, the model picks by textual cues. Differentiate descriptions with active verbs and explicit “Use this when” clauses.
When an agent violates policy, check whether the policy was enforceable. If the forbidden action was possible via a tool without a required guard, the bug is in the schema, not the prompt. Move the guard into the tool.
Decisive takeaway
The system prompts vs tool descriptions question has a clear answer: treat the system prompt as the agent’s constitution and the tool description as its statutory code. Constitution sets values and broad limits; statutory code defines exact procedures and makes illegal states unrepresentable. Write the prompt to be short and stable. Write each tool to be self-contained, side-effect-aware, and schema-strict. Then swap models freely, measure call accuracy, and iterate on description wording rather than rewriting policy. Engineers who respect this boundary ship agents that survive contact with production traffic.