Function calling vs agents: what's the difference

Function calling and AI agents get conflated constantly, but they solve different problems. Function calling is a structured output protocol: the model emits a JSON payload matching a schema you define, and your code executes the corresponding function. An AI agent is a control loop that plans, acts, observes, and iterates — often using function calling as its action primitive. Understanding the distinction changes how you architect LLM-powered systems.

What function calling actually is

Function calling (sometimes called tool use) is a constrained decoding feature. You pass a list of function schemas alongside your prompt. The model responds with a tool_calls array instead of (or in addition to) natural language. Your runtime parses the arguments, invokes the function, and feeds the result back as a tool message.

{
  "name": "get_weather",
  "description": "Get current weather for a location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {"type": "string", "description": "City, state"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
    },
    "required": ["location"]
  }
}

# Your runtime handles the loop
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Weather in Seattle?"}],
    tools=[weather_tool],
    tool_choice="auto"
)

if response.choices[0].message.tool_calls:
    for call in response.choices[0].message.tool_calls:
        result = dispatch(call.function.name, json.loads(call.function.arguments))
        messages.append({"role": "tool", "tool_call_id": call.id, "content": result})
    # Second call to let model incorporate the result
    final = client.chat.completions.create(model="gpt-4o", messages=messages)

Key characteristics: the model decides which function and with what arguments. Your code decides whether to execute, how to execute, and what to do with the result. The model has no memory of prior executions beyond what you stuff into the context window.

What an AI agent actually is

An agent is a runtime that wraps an LLM in a loop: plan → act → observe → reflect → repeat. The LLM proposes actions (usually via function calling), the runtime executes them, and the loop continues until a termination condition — goal achieved, max iterations, error budget exhausted.

# Minimal agent skeleton
class Agent:
    def __init__(self, tools, max_steps=10):
        self.tools = {t.name: t for t in tools}
        self.max_steps = max_steps
    
    def run(self, goal: str) -> str:
        messages = [{"role": "system", "content": "You are an agent..."},
                    {"role": "user", "content": goal}]
        
        for step in range(self.max_steps):
            response = llm(messages, tools=self.tools)
            
            if not response.tool_calls:
                return response.content  # Done
            
            for call in response.tool_calls:
                result = self.tools[call.function.name].execute(
                    json.loads(call.function.arguments)
                )
                messages.append({
                    "role": "tool",
                    "tool_call_id": call.id,
                    "content": result
                })
        
        return "Max steps exceeded"

Agents add: persistent state (scratchpad, memory), planning/decomposition, error recovery (retry, fallback tools), and termination logic. Frameworks like LangGraph, AutoGen, and CrewAI formalize these patterns with graphs, multi-agent orchestration, and human-in-the-loop checkpoints.

Head-to-head comparison

Capabilities: when each shines

Function calling excels at discrete, well-scoped operations: extract structured data, query an API, transform format, validate input. The model acts as a parser with world knowledge. You control the flow, so latency is predictable — one or two round trips.

Agents handle open-ended goals: “reconcile this invoice against the ERP,” “research competitors and draft a battlecard,” “debug this failing test suite.” The value is decomposition: the model breaks the goal into sub-tasks, selects tools per sub-task, and adapts when a tool fails. But you pay in latency (5–20+ LLM calls), token spend, and non-determinism.

Cost model differences

Function calling cost is linear: input tokens + output tokens + your execution time. Easy to estimate. A typical extraction call runs 500–2,000 tokens total.

Agent cost is multiplicative: (planning tokens + Σ step tokens) × iterations. A 10-step research agent can burn 50k–200k tokens. Add reflection loops and you’re in the millions for complex tasks. Budget guards (max tokens, max steps, max dollars) are not optional — they’re production requirements.

# Cost guard example
class BudgetGuard:
    def __init__(self, max_usd=0.50, max_steps=15):
        self.max_usd = max_usd
        self.max_steps = max_steps
        self.spent = 0.0
        self.steps = 0
    
    def check(self, usage) -> bool:
        self.steps += 1
        self.spent += estimate_cost(usage)
        return self.steps <= self.max_steps and self.spent <= self.max_usd

Latency and throughput

Function calling adds one extra round trip (assistant → tool → assistant). P95 typically 2–4 seconds on modern models. Throughput scales horizontally — stateless, no coordination.

Agents are sequential by default. Ten steps at 3 seconds each = 30 seconds minimum. Parallel tool execution helps (LangGraph’s parallel nodes, OpenAI’s parallel_tool_calls), but dependencies between steps limit gains. For user-facing latency, you need streaming intermediate results or async job APIs with webhooks.

Ergonomics and developer experience

Function calling is a thin wrapper. You write the schema, you write the dispatcher, you own the prompt. Debugging is straightforward: log the tool_call, log the result, inspect.

Agents introduce framework opacity. LangGraph’s state graph, AutoGen’s conversation agents, CrewAI’s role-based crews — each has its own mental model. Debugging means inspecting execution traces, not just logs. The learning curve is real. But you gain: visualization, time-travel debugging, checkpoint/resume, and multi-agent patterns out of the box.

# LangGraph checkpointing — resume after crash/human review
from langgraph.checkpoint.sqlite import SqliteSaver

checkpointer = SqliteSaver.from_conn_string("checkpoints.db")
graph = builder.compile(checkpointer=checkpointer)

# Resume from thread_id
config = {"configurable": {"thread_id": "user-123-session-456"}}
for event in graph.stream(inputs, config):
    print(event)

Ecosystem and portability

Function calling is standardized across OpenAI, Anthropic, Google, Mistral, and open models via Ollama/vLLM. The schema format differs slightly (OpenAI uses tools, Anthropic uses tools with input_schema, but the concept is identical). Switching providers means updating the client, not the architecture.

Agent frameworks are less portable. LangGraph ties you to LangChain’s ecosystem. AutoGen has its own message protocol. CrewAI imposes role/task/crew abstractions. Moving between them requires rewriting the orchestration layer. However, all of them ultimately call the same function-calling APIs underneath.

Limits and failure modes

Function calling fails when: the model picks the wrong tool, hallucinates arguments, or refuses to call a tool despite clear intent. Mitigations: strict schemas, few-shot examples, tool_choice: "required", validation + retry loop.

Agents fail in more creative ways: infinite loops (tool A calls tool B calls tool A), compounding errors (bad search → bad summary → bad decision), cost explosion, context window exhaustion from accumulated history. Mitigations: step budgets, summarization checkpoints, circuit breakers, human approval gates for destructive actions.

# Circuit breaker pattern for agent tools
class CircuitBreaker:
    def __init__(self, failure_threshold=3, timeout=60):
        self.failures = 0
        self.threshold = failure_threshold
        self.timeout = timeout
        self.last_failure = 0
    
    def call(self, tool, args):
        if self.failures >= self.threshold:
            if time.time() - self.last_failure < self.timeout:
                raise CircuitOpen("Tool circuit open")
            self.failures = 0  # Half-open
        
        try:
            result = tool.execute(args)
            self.failures = 0
            return result
        except Exception as e:
            self.failures += 1
            self.last_failure = time.time()
            raise

Which to choose

Use function calling when:

• The task is a single, well-defined transformation or lookup
• You need deterministic latency and cost
• You want full control over execution, retries, and fallbacks
• The workflow is already expressed in your application code
• Examples: entity extraction, SQL generation, API parameter mapping, classification, format conversion

Use an agent when:

• The goal requires multi-step reasoning with branching paths
• The sequence of tools isn’t known upfront
• You need planning, backtracking, or dynamic tool selection
• Human-in-the-loop checkpoints are required (approvals, reviews)
• The task benefits from persistent memory across sessions
• Examples: research synthesis, codebase navigation and refactoring, complex troubleshooting, travel booking with constraints, compliance audits

Hybrid approach (often the right answer): Wrap function calling in a thin orchestrator for 80% of cases. Escalate to an agent only for the genuinely open-ended 20%. Your orchestrator becomes a router: structured intent → function pipeline; ambiguous intent → agent with budget guard.

# Router pattern
def handle_request(user_input: str) -> str:
    intent = classify_intent(user_input)  # Fast, cheap classifier
    
    if intent in STRUCTURED_HANDLERS:
        return STRUCTURED_HANDLERS[intent](user_input)
    
    # Ambiguous — spin up agent with strict budget
    return Agent(tools=ALL_TOOLS, max_steps=8, max_usd=0.25).run(user_input)

The distinction matters because it determines where you spend engineering effort: on schemas and dispatchers, or on state machines and observability. Choose the abstraction that matches the problem’s inherent complexity — not the one with the better marketing.