LlamaIndex FunctionAgent tool calling explained

Tool calling turns an LLM from a text generator into a system that can act on the world. This llamaindex functionagent tool calling tutorial walks through building a FunctionAgent that orchestrates Python functions as tools, covering schema definition, execution flow, and failure modes you will hit in production.

Install and configure the LLM

LlamaIndex’s FunctionAgent delegates reasoning to an LLM that supports tool calling. In practice that means an OpenAI-compatible chat model or a local model exposed through a compatible server.

pip install llama-index-core llama-index-llms-openai

Instantiate the model. If you’d rather not pin to a single provider, point the same OpenAI client at n4n.ai’s OpenAI-compatible endpoint—it fronts 240+ models and automatically falls back when a provider is rate-limited or degraded, while metering per-token usage.

from llama_index.llms.openai import OpenAI

llm = OpenAI(
    model="gpt-4o-mini",
    api_key="sk-...",
    # base_url="https://api.n4n.ai/v1"  # optional gateway
)

Use a model that reliably emits function calls. Smaller models often drop arguments or hallucinate parameters; test before shipping. If you self-host, confirm the server returns tool_calls in the OpenAI format, not just text.

Define tools with strict signatures

A tool is just a Python function wrapped by FunctionTool.from_defaults. LlamaIndex infers the JSON schema from type hints and the docstring. Keep signatures explicit and avoid *args or untyped defaults.

from llama_index.core.tools import FunctionTool

def search_invoices(customer_id: int, status: str = "open") -> list[dict]:
    """Retrieve invoices for a customer by ID and optional status."""
    # pretend DB call
    return [{"id": 1, "customer_id": customer_id, "status": status}]

invoice_tool = FunctionTool.from_defaults(fn=search_invoices)

Pitfall: mutable default arguments (like []) break schema inference and create shared state. Use None and initialize inside the function. Also, Pydantic models as parameters are supported but increase complexity; prefer primitive types unless you need nested structures.

Async tools

If your tool hits a network or DB, make it async. FunctionAgent awaits coroutines natively.

async def fetch_user(user_id: int) -> dict:
    """Fetch a user record from the API."""
    # async http call
    return {"id": user_id, "name": "Ada"}

user_tool = FunctionTool.from_defaults(fn=fetch_user)

Override schema and naming

The inferred name is the function name. Override it for clarity:

invoice_tool = FunctionTool.from_defaults(
    fn=search_invoices,
    name="invoice_lookup",
    description="Looks up invoices by customer and status"
)

Explicit descriptions reduce ambiguous calls. The model picks tools based on this text, not your code.

Construct the FunctionAgent

Pass the tools, the LLM, and a system prompt that tells the model when to call tools. The agent loops: model emits a tool call, LlamaIndex executes it, result is fed back, repeat until a final answer.

from llama_index.core.agent.workflow import FunctionAgent

agent = FunctionAgent(
    tools=[invoice_tool, user_tool],
    llm=llm,
    system_prompt="You are a billing assistant. Use tools to fetch data before answering."
)

Tradeoff: FunctionAgent is single-threaded and runs tools sequentially. If you need parallel tool execution, you must implement fan-out yourself or use a different workflow. For most CRUD-style agents, sequential is fine and easier to debug. The system prompt is not a sandbox; it cannot prevent the model from calling a tool, only bias it.

Run the agent and inspect the loop

The run method returns a handler you await. By default it blocks until the agent finishes, but you lose visibility into intermediate steps.

handler = agent.run(input="Get invoices for customer 42")
response = await handler
print(response.response)

To see tool calls, stream events:

async for event in agent.run(input="Get invoices for customer 42").stream_events():
    if hasattr(event, "tool_calls"):
        print("tool call:", event.tool_calls)
    elif hasattr(event, "response"):
        print("token:", event.response)

Common pitfall: assuming the agent stops after one tool call. Complex queries trigger multiple rounds. Set max_iterations to avoid runaway loops and infinite API spend.

agent = FunctionAgent(
    tools=[invoice_tool],
    llm=llm,
    max_iterations=5,
)

The handler also exposes handler.intermediate_steps after completion, useful for logging what was called and in what order.

Handle tool errors gracefully

Tools fail: APIs 500, schemas mismatch, timeouts. Wrap your function bodies in try/except and return a string error. The model can then recover or explain.

def search_invoices(customer_id: int, status: str = "open") -> str:
    try:
        # db call
        return str([{"id": 1, "customer_id": customer_id}])
    except Exception as e:
        return f"ERROR: {e}"

If you raise, FunctionAgent propagates and the whole run fails. Returning an error string is usually better because the LLM can adapt. For async tools, catch with try/except around await.

Timeouts and retries

LlamaIndex does not auto-retry tools. Implement your own backoff inside the function, or accept that a single failure returns the error string. Do not rely on the model to retry more than once or twice; it often gives up.

Streaming output to users

Users expect tokens, not a 10-second pause. Use stream_events and forward ChatResponse chunks to your UI. Tool execution still blocks between tokens, so show a “tool running” indicator when you see a ToolCall event.

from llama_index.core.agent.workflow import ToolCall

async for ev in agent.run(input="...").stream_events():
    if isinstance(ev, ToolCall):
        yield {"type": "status", "msg": f"calling {ev.tool_name}"}
    # forward text

Keep tool outputs small. A 50KB JSON blob returned from a tool gets re-sent to the model on every subsequent iteration, multiplying token cost. Truncate or summarize inside the tool.

Test tools in isolation

The biggest source of agent bugs is the tool, not the model. Write unit tests for each function with mocked I/O.

def test_search_invoices():
    res = search_invoices(42)
    assert "customer_id" in res

Only after tools are green, run end-to-end prompts. This llamaindex functionagent tool calling tutorial recommends a suite of fixed prompts that must produce specific tool calls—cheap regression checks as models update. Store transcripts of successful runs to diff against after library upgrades.

Debugging with logging

Enable LlamaIndex logging to see raw model requests:

import logging
logging.basicConfig(level=logging.DEBUG)

Look for malformed tool_calls or missing required arguments. Many issues trace to a docstring that contradicts the type hint. Keep them aligned.

Production tradeoffs

• Latency: Each tool round-trip adds a full model inference. Batch data needs into one tool if possible.
• Cost: Every intermediate message is sent back to the model. Large tool outputs blow up token usage. Trim responses.
• Safety: FunctionAgent will call any registered tool. Never register a destructive function without a human confirmation layer.
• Model drift: A provider swap (via gateway or direct) can change call formatting. Keep golden tests.

When not to use FunctionAgent

If your flow is a fixed DAG, use a Workflow. If you need long-term memory and many specialized roles, consider a multi-agent system. FunctionAgent shines for open-ended but tool-bounded tasks like “query the DB and summarize.”

This llamaindex functionagent tool calling tutorial gave you the ordered path: configure LLM, define typed tools, wrap in FunctionAgent, stream events, guard errors, and test. The pattern is stable across LlamaIndex versions, but always pin your major version—the agent API shifted significantly between 0.10 and 0.11.