Writing prompts that reduce hallucinated tool calls

Poorly specified tool schemas cause models to invent parameters or call functions that don’t exist. To reduce hallucinated tool calls prompting, you need to constrain the model’s decision surface with explicit contracts, negative examples, and runtime validation. This guide walks through a concrete pipeline you can ship today.

Step 1: Define strict JSON schemas for every tool

Don’t hand the model a natural-language description and hope it behaves. Emit a JSON Schema with required, enum, and typed properties. Avoid type: "string" with no constraints—use enum or pattern when possible. Set additionalProperties: false so the model can’t smuggle in extra keys.

tools = [
  {
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Fetch current weather for a US zip code.",
      "parameters": {
        "type": "object",
        "additionalProperties": False,
        "properties": {
          "zip": {
            "type": "string",
            "pattern": "^[0-9]{5}$",
            "description": "Five-digit US zip code"
          },
          "unit": {
            "type": "string",
            "enum": ["fahrenheit", "celsius"]
          }
        },
        "required": ["zip"]
      }
    }
  }
]

A model can’t hallucinate a city parameter if it isn’t in the schema. The schema is the first line of defense. I’ve seen agents drop 30% of invalid calls just by removing additionalProperties: true and adding pattern checks.

If your tool takes an entity ID, validate it against a known prefix. If it takes a date, use format: "date" rather than a free string. The tighter the schema, the less room for speculation.

Step 2: Inject explicit negative examples in the system prompt

Models copy patterns from context. If you show only positive calls, they assume calling is always expected. Add a short “do not call” section that shows the silent path.

You have access to tools for fetching weather and booking meetings.
Rules:
- Only call a tool when the user explicitly provides the required parameters.
- If the user asks a general question (e.g., "what's the weather like?") without a location, DO NOT call get_weather. Reply with a clarifying question.
- Never invent parameters. If a value is missing, ask.
- If the user greets you or makes small talk, respond with text only.

This type of reduce hallucinated tool calls prompting works because it sets a prior: silence is allowed. The default behavior of most instruction-tuned models is to be helpful by acting; you must explicitly license inaction.

Keep the negative examples concrete. “Don’t call tools” is weaker than “If zip is missing, ask for it; do not call get_weather.” The latter maps directly to a schema violation the model can anticipate.

Step 3: Force a plan-then-call loop

Many hallucinations happen because the model emits a tool call in the same breath as ambiguous input. Require it to state intent first. With the OpenAI API you can use a two-step pattern: a planning turn with tools=[], then a real turn.

import openai

def plan_and_call(client, messages):
    # Step A: ask for a plan only, no tools available
    plan_resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages + [{"role": "system", "content": "Output a one-line plan. No tool calls."}],
        tools=[],
    )
    plan = plan_resp.choices[0].message.content
    messages.append({"role": "assistant", "content": plan})
    # Step B: now allow tools
    call_resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        tools=tools,
        tool_choice="auto",
    )
    return call_resp

The plan step surfaces missing params as text, not as a malformed call. You can also enforce this with a structured output schema for the plan, but plain text is usually enough to break the reflex.

Set temperature low (0.2 or below) for the call step. High temperature increases creative parameter fabrication. For agents, determinism on the action boundary matters more than linguistic flair.

Step 4: Narrow the tool set per turn

Don’t send all 40 tools when the conversation is about weather. Filter by intent or prior state. A smaller candidate set linearly reduces the chance of a wrong pick.

def filter_tools(state):
    if state.get("topic") == "weather":
        return [t for t in tools if t["function"]["name"].startswith("get_weather")]
    if state.get("topic") == "calendar":
        return [t for t in tools if "book" in t["function"]["name"]]
    return tools

In a state-machine agent, compute the allowed tools from the current node. This is a cheap, high-leverage change. I’ve measured 4x fewer misroutes after moving from a static 25-tool list to a dynamic 2–3 tool list per step.

If you use tool_choice, prefer "auto" over forcing a specific function. Forcing invites calls when none are warranted.

Step 5: Validate at the boundary and reject

Never trust the model’s JSON. Validate against the schema and return a tool error if it fails. The model can self-correct when given the error.

import json, jsonschema

def safe_invoke(call):
    fn = call.function.name
    try:
        args = json.loads(call.function.arguments)
    except json.JSONDecodeError:
        return {"role": "tool", "content": "Args not valid JSON", "tool_call_id": call.id}
    schema = next(t for t in tools if t["function"]["name"] == fn)["function"]["parameters"]
    try:
        jsonschema.validate(args, schema)
    except jsonschema.ValidationError as e:
        return {"role": "tool", "content": f"Invalid args: {e.message}", "tool_call_id": call.id}
    # real invocation here, guarded by idempotency key
    return {"role": "tool", "content": "ok", "tool_call_id": call.id}

This closes the loop: bad calls become feedback, not silent failures. Pair validation with an idempotency key per tool_call_id so retries don’t double-book or double-fetch.

If you see the same invalid call repeated after the error, your schema or prompt is contradictory. Fix the contract, don’t just loop harder.

Step 6: Run a cross-model evaluation harness

Build 50 ambiguous prompts (“weather?”, “book something”, “hello”) and measure how often the model either calls without params or invents a tool. Script it against multiple models to see which ones need stricter prompts.

from openai import OpenAI
client = OpenAI(base_url="https://api.n4n.ai/v1")  # one OpenAI-compatible endpoint, 240+ models

models_to_test = ["gpt-4o-mini", "claude-3-haiku", "mistral-small"]
ambiguous_prompts = ["weather?", "book something", "hi", "what's the temp?"]
for model in models_to_test:
    bad = 0
    for prompt in ambiguous_prompts:
        resp = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            tools=tools,
        )
        if resp.choices[0].message.tool_calls:
            bad += 1
    print(model, bad/len(ambiguous_prompts))

Using a gateway like n4n.ai lets you run the same reduce hallucinated tool calls prompting across providers without rewriting your client or managing separate keys. The per-token metering also shows which models cost more to keep honest, which informs your routing rules.

Always include a “no-op” baseline in the harness: a prompt that should never trigger a call. If any model calls, your prompt is leaky.

How to verify success

Success means three things in production:

1. Zero schema violations after boundary validation—every call that reaches your backend passes jsonschema.
2. Clarification appears instead of blind calls on missing params. Sample 100 ambiguous logs; the model should ask, not call.
3. Eval harness shows <5% invalid-call rate on the ambiguous set across your target models.

Log every tool_call with its prompt hash and model ID. If the rate climbs after a model swap, tighten the system prompt or shrink the tool list before reaching for higher temperature cuts. The techniques above are not theoretical; they reflect what works in production agents handling thousands of turns a day.