A beginner's guide to LLM tool use

If you’re building with LLMs today, tool use (often called function calling) is the difference between a chatbot and an agent that can actually do work. This beginner’s guide to LLM tool use covers the mechanics, patterns, and pitfalls you’ll hit when moving from prototype to production. We’ll start with the request/response cycle, move through schema design and execution patterns, and finish with the operational concerns that bite teams in production.

The request/response cycle

Tool use follows a predictable loop: the model proposes a tool call, your code executes it, you return the result, and the model continues. Most providers implement this as a multi-turn conversation with special message roles.

User: "What's the weather in Tokyo?"
Assistant: [tool_call: get_weather(location="Tokyo")]
Tool: {"temperature": 18, "condition": "cloudy"}
Assistant: "It's 18°C and cloudy in Tokyo."

The model never executes code. It only emits structured arguments matching a schema you provide. Your runtime — whether that’s a simple script, a framework like LangChain, or a gateway — is responsible for dispatching, timeout handling, and feeding results back.

OpenAI-compatible APIs expose this through the tools parameter (an array of function schemas) and tool_choice (auto, required, none, or a specific function name). Anthropic uses a similar tools array but returns tool calls in a tool_use block within the message content. The wire format differs; the mental model is the same.

# Minimal OpenAI-compatible tool call request
import openai

client = openai.OpenAI()

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City name"}
            },
            "required": ["location"]
        }
    }
}]

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Weather in Tokyo?"}],
    tools=tools,
    tool_choice="auto"
)

# response.choices[0].message.tool_calls contains the proposed calls

Schema design: the contract between model and code

Your JSON Schema is the API contract. Models are surprisingly good at following schemas when they’re clear, and surprisingly creative at breaking them when they’re not.

Be specific with descriptions. The model reads your property descriptions to understand intent. "location": {"type": "string"} is ambiguous. "location": {"type": "string", "description": "City name, e.g. 'Tokyo' or 'San Francisco'"} works better.

Constrain enums aggressively. If a parameter accepts only “celsius” or “fahrenheit”, declare it as an enum. The model will hallucinate “metric” and “imperial” otherwise.

Avoid optional parameters with complex defaults. Required parameters force the model to think about what it needs. Optional parameters with implicit defaults (e.g., “units defaults to celsius”) lead to silent failures where the model omits the field and your code assumes a default that changed.

{
  "name": "search_docs",
  "description": "Search internal documentation",
  "parameters": {
    "type": "object",
    "properties": {
      "query": {"type": "string", "description": "Search query, 3-10 words"},
      "collection": {
        "type": "string",
        "enum": ["api-reference", "guides", "changelog"],
        "description": "Which documentation collection to search"
      },
      "max_results": {"type": "integer", "minimum": 1, "maximum": 20, "default": 5}
    },
    "required": ["query", "collection"],
    "additionalProperties": false
  }
}

Set additionalProperties: false. Without it, models invent parameters that don’t exist. This is the single most common schema bug.

Execution patterns

Sequential (default)

The model calls one tool, you return the result, the model decides the next step. Simple, debuggable, but slow for independent operations.

Parallel tool calls

When the model needs multiple independent pieces of information, it can emit several tool calls in a single response. You execute them concurrently and return all results before the next model turn.

# Handling parallel tool calls
tool_calls = response.choices[0].message.tool_calls

# Execute all in parallel
import asyncio

async def execute_tool(call):
    name = call.function.name
    args = json.loads(call.function.arguments)
    result = await dispatch(name, args)  # your dispatch logic
    return {
        "role": "tool",
        "tool_call_id": call.id,
        "content": json.dumps(result)
    }

tool_results = await asyncio.gather(*[execute_tool(tc) for tc in tool_calls])

# Feed all results back in one turn
followup = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        *messages,
        response.choices[0].message,
        *tool_results
    ],
    tools=tools
)

Parallel calls cut latency dramatically for fan-out patterns (searching multiple collections, checking several APIs). The tradeoff: you lose the model’s ability to reason between steps. If tool B depends on tool A’s output, don’t parallelize.

Structured output / tool choice “required”

Sometimes you want the model to only emit a tool call — no conversational filler. Set tool_choice: {"type": "function", "function": {"name": "your_function"}}. This is useful for extraction pipelines where the model’s job is to populate a schema, not chat.

Error handling the model can understand

Tools fail. Networks timeout. APIs return 429. Your job is to return errors in a format the model can reason about, not crash the conversation.

Return structured errors, not exceptions. The model sees the tool result message. If you throw an uncaught exception, the conversation breaks. Instead, return a result object the model can interpret:

async def dispatch(name: str, args: dict) -> dict:
    try:
        result = await TOOL_REGISTRY[name](**args)
        return {"success": True, "data": result}
    except RateLimitError as e:
        return {
            "success": False,
            "error": "rate_limited",
            "message": f"API rate limit hit. Retry after {e.retry_after_seconds}s",
            "retry_after": e.retry_after_seconds
        }
    except ValidationError as e:
        return {
            "success": False,
            "error": "invalid_args",
            "message": str(e)
        }
    except Exception as e:
        # Log the full traceback for debugging
        logger.exception(f"Tool {name} failed")
        return {
            "success": False,
            "error": "internal_error",
            "message": "Tool execution failed. Try a different approach."
        }

The model can now see {"success": false, "error": "rate_limited", ...} and decide to retry, switch tools, or tell the user to wait. This is far more useful than a stack trace in the chat.

Idempotency matters. If a tool call times out and the model retries, you’ll execute twice. Design mutating tools (send_email, create_record, charge_card) to be idempotent — accept an idempotency_key the model can generate or you can derive from the conversation context.

Common pitfalls

The model ignores your tool

Causes: vague description, schema too complex, model genuinely doesn’t need the tool for this query. Fix: simplify the schema, sharpen the description, or accept that tool_choice: "auto" means the model decides.

The model hallucinates arguments

Causes: insufficient constraints, enum missing valid values, model confused by similar parameter names. Fix: add enums, tighten descriptions, use distinct parameter names across tools.

Infinite tool loops

The model calls a tool, gets a result, calls the same tool again with slightly different args, repeats. Fix: implement a max-turns guard in your orchestration loop, and design tools to return enough context that the model doesn’t need to re-query.

MAX_TOOL_TURNS = 8

for turn in range(MAX_TOOL_TURNS):
    response = client.chat.completions.create(...)
    if not response.choices[0].message.tool_calls:
        break
    # execute tools, append results
else:
    # Force a final response without tools
    final = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages + [{"role": "system", "content": "Stop using tools. Answer now."}],
        tools=tools,
        tool_choice="none"
    )

Token bloat from tool results

Large tool results (full HTML pages, huge JSON blobs) consume context window and cost money. Fix: summarize in the tool before returning, or implement a truncation policy.

def truncate_result(result: dict, max_chars: int = 3000) -> dict:
    content = json.dumps(result)
    if len(content) <= max_chars:
        return result
    return {
        "truncated": True,
        "original_size": len(content),
        "summary": f"Result truncated from {len(content)} chars. Key fields: {list(result.keys())[:10]}"
    }

Production patterns

Tool registry with versioning

Don’t hardcode tool definitions in your prompt construction. Maintain a registry that serves schemas to the model and maps names to executors. Version your schemas so you can roll out changes without breaking in-flight conversations.

# tools/registry.py
from dataclasses import dataclass
from typing import Callable, Awaitable
import json

@dataclass
class Tool:
    name: str
    schema: dict
    executor: Callable[..., Awaitable[dict]]
    version: str = "1"

REGISTRY: dict[str, Tool] = {}

def register(tool: Tool):
    REGISTRY[tool.name] = tool

def get_schemas() -> list[dict]:
    return [t.schema for t in REGISTRY.values()]

async def execute(name: str, args: dict) -> dict:
    tool = REGISTRY.get(name)
    if not tool:
        return {"success": False, "error": "unknown_tool", "message": f"No tool named {name}"}
    return await tool.executor(**args)

Observability: log every tool call

You need to know which tools are called, with what arguments, latency, success rate, and token cost. At minimum, log:

Tool name and version
Input arguments (redact secrets)
Execution latency
Success/failure and error type
Tokens consumed by the tool result message

This lets you answer “why did this conversation cost $0.40?” and “why is get_weather failing 15% of the time?”

Fallback and degradation

If a tool provider is down, your agent should degrade gracefully. Options:

Return a cached/stale result with a freshness warning
Route to an alternative tool (e.g., fallback from a premium search API to a basic one)
Tell the model the tool is unavailable so it can work around it

Some gateways handle provider-level fallback automatically — for instance, n4n.ai routes around degraded providers while preserving the tool call interface so your code doesn’t need to know which upstream model actually executed.

Authentication and authorization

Tools often need credentials (API keys, OAuth tokens, database connections). Don’t bake these into tool executors. Pass a context object with the authenticated client or token, scoped to the user or session.

async def get_user_docs(query: str, *, ctx: RequestContext) -> dict:
    # ctx carries user_id, auth_token, rate_limit_bucket, etc.
    client = ctx.get_api_client("docs")
    return await client.search(query, user_id=ctx.user_id)

This keeps tools pure and testable, and lets you swap auth strategies without touching tool logic.

Testing tool use

Unit test your executors in isolation. Integration test the full loop with a fixed model response (use a mock or recorded fixture). Key scenarios:

Happy path: model calls tool, tool succeeds, model produces final answer
Tool error: model calls tool, tool returns structured error, model recovers
Parallel calls: model emits 3 tool calls, all execute concurrently, results aggregated
Max turns: model loops, orchestrator cuts off at limit, forces final answer
Schema validation: model emits invalid args, your dispatcher catches and returns validation error

# tests/test_tool_loop.py
import pytest
from unittest.mock import AsyncMock, patch

@pytest.mark.asyncio
async def test_weather_tool_success():
    with patch("openai.OpenAI") as mock_client:
        # Mock model proposing tool call
        mock_client.return_value.chat.completions.create.side_effect = [
            # First call: model proposes tool
            Mock(choices=[Mock(message=Mock(
                tool_calls=[Mock(id="call_1", function=Mock(name="get_weather", arguments='{"location": "Tokyo"}'))]
            ))]),
            # Second call: model responds with result
            Mock(choices=[Mock(message=Mock(content="It's 18°C in Tokyo."))])
        ]
        
        result = await run_agent("Weather in Tokyo?")
        assert "18°C" in result
        assert "Tokyo" in result

When not to use tools

Tools add latency, complexity, and failure modes. Don’t reach for them when:

The model can answer from training data (facts, coding, reasoning)
A simple RAG retrieval + prompt stuffing works (static knowledge bases)
The operation is better expressed as a deterministic function the caller runs before invoking the model (e.g., “get current user’s ID” — just pass it in the prompt)

Tools shine when the model needs to decide what to do based on the conversation: which API to call, what parameters to use, whether to retry, how to combine results. That’s the agent boundary.

Summary checklist

Tool use is the bridge between LLM reasoning and real-world effects. Get the contract right, handle failures gracefully, and instrument everything. The rest is iteration.