How AI agents use tools to complete tasks

An AI agent is a system that uses a language model to reason about a task, select tools from a defined set, execute them, and incorporate the results into subsequent reasoning steps. The model does not merely produce text; it emits structured calls that trigger code execution, API requests, or data lookups, then decides what to do next based on what those tools return. This loop — reason, act, observe, repeat — is the minimal viable definition of agentic behavior.

How the tool-use loop works

At the API level, tool use is function calling with a standardized contract. The developer provides a JSON Schema for each tool: name, description, and parameter schema. The model, when prompted with a user goal and the tool catalog, can emit a tool_calls array instead of (or alongside) a text response. The host application executes each call, captures the result, and feeds it back as a tool role message. The model then continues.

{
  "name": "get_weather",
  "description": "Return current conditions for a latitude/longitude.",
  "parameters": {
    "type": "object",
    "properties": {
      "lat": { "type": "number" },
      "lon": { "type": "number" }
    },
    "required": ["lat", "lon"]
  }
}

A minimal execution loop in Python looks like this:

import json
from openai import OpenAI

client = OpenAI()
tools = [{"type": "function", "function": get_weather_schema}]

messages = [
    {"role": "system", "content": "You have access to get_weather. Use it when needed."},
    {"role": "user", "content": "What's the weather in San Francisco?"}
]

while True:
    resp = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools,
        tool_choice="auto"
    )
    msg = resp.choices[0].message
    messages.append(msg.model_dump())

    if not msg.tool_calls:
        break

    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(**args)  # your implementation
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result)
        })

print(messages[-1]["content"])

The loop terminates when the model emits a final answer with no tool_calls. Production systems add timeouts, max-iteration guards, and structured output validation — but the skeleton is always this.

Tool categories engineers actually use

Each category demands different safety rails. Mutation tools need idempotency keys and confirmation prompts. Computation tools need resource limits and deterministic seeds. Orchestration tools need circuit breakers and distributed tracing.

Why tool use changes what you can build

Without tools, a model is a closed-book exam: it can only recall training data or manipulate context you stuff into the prompt. With tools, it becomes an open-book exam where the book is your entire infrastructure.

Three capabilities emerge:

1. Grounded answers — The model queries your live database instead of hallucinating schema. A support bot checks order status via your actual orders API, not a RAG index that lagged by six hours.
2. Multi-step workflows — The model can decompose “refund the last order and email the customer” into: lookup order → verify eligibility → call refund API → compose email → send via SendGrid. Each step conditions on the previous result.
3. Dynamic tool selection — Given a catalog of 50 tools, the model picks the relevant subset per task. You don’t hardcode a DAG; the model plans at inference time.

This shifts engineering effort from prompt engineering to tool design: clear schemas, helpful descriptions, predictable error shapes, and observability hooks.

Concrete example: a coding agent that fixes a failing test

Consider a repository with a failing pytest suite. The agent’s goal: make the tests pass. Tools available:

tools = [
    {"type": "function", "function": {
        "name": "read_file",
        "description": "Return file contents as text",
        "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}
    }},
    {"type": "function", "function": {
        "name": "write_file",
        "description": "Overwrite a file",
        "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}}, "required": ["path", "content"]}
    }},
    {"type": "function", "function": {
        "name": "run_tests",
        "description": "Run pytest and return stdout/stderr/exit_code",
        "parameters": {"type": "object", "properties": {"args": {"type": "array", "items": {"type": "string"}}}, "required": ["args"]}
    }},
    {"type": "function", "function": {
        "name": "grep",
        "description": "Search codebase for pattern",
        "parameters": {"type": "object", "properties": {"pattern": {"type": "string"}}, "required": ["pattern"]}
    }}
]

A typical trace:

1. User: “Fix the failing tests in tests/auth_test.py.”
2. Model → run_tests(["tests/auth_test.py"]) → sees AssertionError: expected 201, got 401 on test_create_user.
3. Model → read_file("tests/auth_test.py") → sees the test expects a JWT on success.
4. Model → grep("create_user") → finds handler in app/auth.py.
5. Model → read_file("app/auth.py") → spots missing return token after user creation.
6. Model → write_file("app/auth.py", fixed_content).
7. Model → run_tests(["tests/auth_test.py"]) → passes.
8. Model → final answer summarizing the fix.

No human wrote a plan. The model explored, hypothesized, edited, verified. The tools were the only way it touched the repository.

Common misconceptions

“Agents are just ReAct prompting”

ReAct (Reason + Act) is a prompting pattern: Thought: ... Action: ... Observation: .... Modern function calling replaces the text-based Action: parsing with structured tool_calls. The loop is the same; the reliability is not. Structured calls eliminate parsing failures, enable parallel invocation, and let the provider enforce schema compliance. If you’re still regexing Action: search("foo") from model output, you’re building on a deprecated pattern.

“More tools = smarter agent”

Tool count correlates negatively with reliability beyond a threshold. Each tool adds:

• Token overhead in the system prompt
• Ambiguity for the model (which of these three similar tools?)
• Surface area for hallucinated parameters

Prefer composable primitives over task-specific tools. One http_request tool with method/url/headers/body beats get_user, create_order, cancel_subscription, update_profile. The model assembles the HTTP call; you maintain one execution path with shared auth, retries, logging, and rate limiting.

“The model plans perfectly if you give it good tools”

Models still:

• Forget to call a tool they needed
• Call tools in the wrong order
• Misread tool results (especially large JSON)
• Get stuck in loops (call A → result → call A again with same args)

Mitigations:

• Step budgets: hard limit on iterations (e.g., 10)
• Result summarization: truncate or embed large tool outputs before feeding back
• Explicit reflection: add a reflect tool that forces the model to write a scratchpad before the next action
• Human-in-the-loop gates: require approval for mutation tools

# Reflection tool example
reflect_tool = {
    "type": "function",
    "function": {
        "name": "reflect",
        "description": "Write internal reasoning before next action. Not shown to user.",
        "parameters": {
            "type": "object",
            "properties": {
                "situation": {"type": "string"},
                "hypothesis": {"type": "string"},
                "next_step": {"type": "string"}
            },
            "required": ["situation", "hypothesis", "next_step"]
        }
    }
}

“Tool use requires a fancy agent framework”

LangGraph, Autogen, CrewAI, and others provide orchestration scaffolds — state graphs, multi-agent handoffs, memory persistence. They are useful when you need them. But the core loop is 30 lines of code (see above). Start there. Adopt a framework when you hit a specific pain point: checkpointing long runs, visual debugging, or multi-agent coordination. Don’t import a framework to avoid understanding the loop.

Designing tools that models can actually use

Schema clarity beats completeness

A tool with 20 optional parameters confuses the model. Split it. search(query, filters?) is better than search(query, filters?, sort?, page?, page_size?, highlight?, facet?). If callers need pagination, make a search_next(page_token) tool.

Error shapes must be machine-readable

Never return plain-text errors. Return structured payloads the model can reason about:

{
  "error": {
    "code": "RATE_LIMITED",
    "retry_after_ms": 1200,
    "message": "Provider quota exceeded"
  }
}

The model can then decide: wait and retry, try a fallback tool, or escalate to the user. A string "rate limited" forces the model to guess.

Descriptions are prompt engineering

The description field in your JSON Schema is the only documentation the model sees. Write it like a man page:

"description": "Execute a SELECT query. Returns rows as array of objects. Fails if query modifies data. Max 1000 rows; use LIMIT."

Not: "Run SQL queries."

Idempotency for every mutation

Any tool that changes state must accept an idempotency_key (client-generated UUID) and return the same result on retries. The model will retry on network blips. Your database must handle it.

async def create_order(args, idempotency_key: str):
    async with db.transaction():
        existing = await db.fetchone(
            "SELECT * FROM orders WHERE idempotency_key = $1", idempotency_key
        )
        if existing:
            return existing
        return await db.execute("INSERT ...", args, idempotency_key)

Observability you’ll wish you had

Log every loop iteration as a structured event:

{
  "timestamp": "2025-01-15T12:34:56.789Z",
  "run_id": "run_abc123",
  "iteration": 3,
  "model": "gpt-4o",
  "tool_calls": [
    {"name": "http_request", "args": {"method": "GET", "url": "https://api.example.com/users/42"}}
  ],
  "tool_results": [
    {"name": "http_request", "status": 200, "latency_ms": 142, "response_bytes": 892}
  ],
  "tokens_in": 3421,
  "tokens_out": 512
}

With this you can:

• Detect loops (same tool, same args, repeated)
• Measure tool latency percentiles
• Correlate token spend with task success
• Replay failures locally

When to use an inference gateway

If you route across multiple providers (OpenAI, Anthropic, open-weight models on Together, Fireworks, etc.), tool schemas diverge. OpenAI uses tools with function objects. Anthropic uses tools with input_schema. Some open-weight models expect a functions parameter or a bespoke chat template. A gateway that normalizes these differences — one OpenAI-compatible request in, provider-appropriate payload out — eliminates a class of integration bugs. It also lets you swap models for cost or latency without rewriting tool definitions.

Summary

Tool use turns a language model into an actor. The mechanism is a structured function-calling loop: the model proposes calls, your code executes them, results feed back, the model decides next. The engineering leverage lives in tool design — small, composable, well-described, idempotent, observable. Frameworks help at scale; the loop itself is simple. Build the loop first, instrument it ruthlessly, and treat every tool as a production API surface — because that’s exactly what it is.