Function calling schemas: JSON Schema for LLM tools

Defining a robust json schema for llm function calling is the difference between a tool that works reliably and one that silently drops parameters. This guide walks through the concrete steps to author tool definitions that survive contact with real models and gateways, from basic structure to provider-specific quirks.

1. Anchor on the OpenAI tool schema shape

Most LLM providers accept an OpenAI-compatible tools array. Each entry has type: "function", a function object with name, description, and parameters. The parameters field is a JSON Schema object (typically a subset of draft-07 or 2020-12).

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Fetch current weather for a location",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {"type": "string"},
        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
      },
      "required": ["location"]
    }
  }
}

Treat description as a contract, not documentation. Models read it to decide when to call. Be explicit about side effects and required context.

2. Use strict typing and required early

The json schema for llm function calling must constrain inputs. Set additionalProperties: false on objects to prevent the model from inventing fields. Mark critical fields required.

{
  "type": "object",
  "properties": {
    "user_id": {"type": "integer"},
    "notify": {"type": "boolean"}
  },
  "required": ["user_id"],
  "additionalProperties": false
}

Omitting additionalProperties: false leads to silent passthrough of garbage in some runtimes. Strict mode reduces hallucinated keys but may cause provider-side validation errors if the model adds a benign extra field—handle that in your gateway.

Enums beat free strings

If a parameter has a fixed set of values, use enum. Models respect enums more reliably than natural language instructions in the description.

"status": {"type": "string", "enum": ["open", "closed", "pending"]}

3. Model nested structures explicitly

Real tools need addresses, filters, or config blobs. Define nested objects with their own properties and required. Avoid type: "object" with no shape; the model will guess.

{
  "type": "object",
  "properties": {
    "address": {
      "type": "object",
      "properties": {
        "street": {"type": "string"},
        "city": {"type": "string"},
        "zip": {"type": "string", "pattern": "^[0-9]{5}$"}
      },
      "required": ["street", "city"],
      "additionalProperties": false
    }
  }
}

Arrays need items. Specify the item schema, not just type: "array".

"tags": {"type": "array", "items": {"type": "string", "maxLength": 32}}

Avoid deep recursion in schemas. Some providers flatten or reject $ref cycles. Inline small structures; extract shared pieces only if your tooling supports $defs reliably.

4. Apply numeric and string constraints

Use minimum, maximum, minLength, maxLength, pattern. These guard your backend without extra validation code. But note: some models ignore pattern for generation, so validate server-side.

{
  "age": {"type": "integer", "minimum": 0, "maximum": 120},
  "email": {"type": "string", "format": "email"}
}

format is advisory; do not rely on it for security. The same applies to exclusiveMinimum or multipleOf—they are hints, not guarantees.

5. Avoid ambiguous and overlapping schemas

If you expose two functions with similar names or parameters, the model may call the wrong one. Differentiate via description and distinct required fields. Do not define type: ["string", "null"] unless your backend handles nulls; many providers flatten unions poorly.

OneOf and anyOf are fragile

Schema composition keywords like oneOf often confuse inference. Prefer a single flat object with optional fields. If you need polymorphism, use a discriminator string and validate after the call.

{
  "type": "object",
  "properties": {
    "action": {"type": "string", "enum": ["create", "delete"]},
    "resource_id": {"type": "string"}
  },
  "required": ["action"]
}

6. Test the schema against real model outputs

Authoring is half the work. Send the tool definition to a few models and inspect raw arguments. Write a small harness:

import openai

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Fetch current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

resp = openai.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What's the weather in NY?"}],
    tools=tools,
)
print(resp.choices[0].message.tool_calls)

Check that required fields appear and enums are respected. If a model repeatedly misses a field, rename it or move it to required. Run the same test across at least two model families; behavior diverges.

7. Provider and gateway compatibility

Not all providers implement the same JSON Schema subset. Some reject additionalProperties: false at top level; others ignore minLength. When you route through a gateway like n4n.ai, the endpoint forwards your schema to 240+ models and handles fallback if a provider errors on validation. Still, target the lowest common denominator: draft-07 core keywords only.

Honor cache-control hints if your gateway supports them; stable tool schemas improve provider-side caching. Per-token metering means verbose descriptions cost money on every call—keep them tight.

8. Common pitfalls and tradeoffs

• Over-specifying: Too many required fields forces the model to ask the user or abort. Balance strictness with usability.
• Under-specifying: Loose schemas shift validation burden to your code and produce runtime errors.
• Description drift: Changing a description without updating backend logic breaks calls. Treat schema as API versioning.
• Large schemas: Every token in the schema eats context window. A 2KB tool def across 10 tools is 20KB of prompt overhead.

Tradeoff: strict vs. flexible

Strict additionalProperties: false prevents junk but may cause the call to fail validation at the provider if the model adds a benign key. Flexible accepts noise but risks injection. We default to strict for internal tools, flexible for user-facing assistants where partial progress is better than failure.

9. Validate before you execute

Never trust the model’s output. Run the arguments through a validator (e.g., jsonschema in Python) before hitting your API.

from jsonschema import validate, ValidationError

try:
    validate(instance=args, schema=parameters_schema)
except ValidationError as e:
    return {"error": "invalid arguments", "detail": e.message}

This decouples model quality from system reliability. Return a structured error to the model so it can retry with corrected arguments.

10. Iterate with real traffic

Log raw tool calls. Cluster failures: missing fields, enum violations, type mismatches. Update the json schema for llm function calling accordingly. Schema is a living contract between your code and the model.

Keep definitions in version control. Review changes like you would any API PR. A diff that adds a required field is a breaking change for existing prompts—tag it and communicate.

Start strict, test broad, validate always. That loop is the only reliable way to ship LLM tools that don’t surprise you in production.