Tool-calling agents with LlamaIndex and Claude Opus 4.8

Building a tool-calling agent that pairs LlamaIndex with Claude Opus 4.8 is straightforward once you understand how LlamaIndex serializes Python functions into model-facing schemas. This tutorial walks through a runnable setup for llamaindex tool calling claude using the native Anthropic integration, showing how to define tools, wire up the agent loop, and inspect the intermediate calls.

Prerequisites

• Python 3.11 or newer
• An Anthropic API key with access to claude-opus-4-8 (or a gateway that exposes it)
• Familiarity with asyncio and basic LlamaIndex concepts

If you plan to route through an OpenRouter-class gateway, the same tool schemas work unchanged; more on that below.

Install dependencies

Create a clean virtual environment and install the required packages.

python -m venv .venv
source .venv/bin/activate
pip install "llama-index>=0.11" llama-index-llms-anthropic

Set your API key:

export ANTHROPIC_API_KEY="sk-ant-..."

Define your tools

LlamaIndex converts a typed Python function into a FunctionTool with a JSON schema. Keep signatures explicit—Claude uses the parameter names and type hints to build its tool spec.

from llama_index.core.tools import FunctionTool

def get_weather(city: str) -> str:
    """Return the current weather summary for a city."""
    # Stubbed; replace with a real API call.
    return f"Clear skies, 22°C in {city}"

def multiply(a: int, b: int) -> int:
    """Multiply two integers and return the product."""
    return a * b

weather_tool = FunctionTool.from_defaults(fn=get_weather)
math_tool = FunctionTool.from_defaults(fn=multiply)

The docstrings matter. Claude Opus 4.8 reads them to decide when to call a tool, so write them as imperative descriptions of what the function does.

Configure the Claude LLM

Instantiate the Anthropic LLM with the exact model string your provider expects.

from llama_index.llms.anthropic import Anthropic

llm = Anthropic(
    model="claude-opus-4-8",
    api_key=os.environ["ANTHROPIC_API_KEY"],
    max_tokens=1024,
)

If you route inference through n4n.ai, the same OpenAI-compatible endpoint forwards provider cache-control hints and honors your routing directives—swap in the OpenAI client with base_url set to the gateway and keep the tool list identical. No LlamaIndex code changes required.

Build the FunctionAgent

LlamaIndex’s FunctionAgent (from the workflow module) handles the ReAct-style loop natively. It sends the tools to Claude, parses the tool_use blocks, executes the local functions, and feeds results back.

from llama_index.core.agent.workflow import FunctionAgent

agent = FunctionAgent(
    tools=[weather_tool, math_tool],
    llm=llm,
    system_prompt="You are a precise assistant. Use tools when needed and show final answers concisely.",
)

Run a single-tool query

Wrap the call in an async main and print the final response.

import asyncio

async def main():
    response = await agent.run("What is the weather in Lisbon?")
    print("Final answer:", response.response)

asyncio.run(main())

Expected output:

Final answer: Clear skies, 22°C in Lisbon

Under the hood Claude emitted a single tool_use block for get_weather with {"city": "Lisbon"}. The agent executed the stub and returned the string.

Multi-tool orchestration

The real value of llamaindex tool calling claude shows when the model chains tools across turns.

async def multi():
    response = await agent.run(
        "What is 17 times the temperature in Celsius for Lisbon? "
        "Assume the weather tool returns the temperature as the number before the C."
    )
    print(response.response)

asyncio.run(multi())

A competent run produces something like:

Final answer: 374 (17 × 22)

Trace the loop and you’ll see two tool_use events: first get_weather, then multiply with a=17, b=22. The agent’s chat_history exposes these steps if you need to log them:

for msg in response.chat_history:
    if msg.role == "assistant" and msg.additional_kwargs.get("tool_calls"):
        print(msg.additional_kwargs["tool_calls"])

Streaming intermediate steps

For production UIs you rarely want to block on the full loop. Use the agent’s astream_step interface to yield events.

from llama_index.core.agent.workflow import ToolCallResult

async def stream_run():
    handler = agent.run("Multiply 42 by the Lisbon temperature")
    async for event in handler.stream_events():
        if isinstance(event, ToolCallResult):
            print(f"Tool {event.tool_name} returned {event.tool_output}")

asyncio.run(stream_run())

This prints each tool result as it resolves, letting you render progress without waiting for Claude’s final synthesis.

Error handling and guardrails

Tool functions throw. Wrap them so the agent gets a clean string instead of a stack trace that leaks internals.

def get_weather(city: str) -> str:
    try:
        # real call here
        return f"Clear skies, 22°C in {city}"
    except Exception as e:
        return f"ERROR: weather lookup failed: {e}"

Claude will typically retry with corrected arguments or report the failure to the user. Set max_iterations on the agent to bound runaway loops:

agent = FunctionAgent(
    tools=[weather_tool, math_tool],
    llm=llm,
    max_iterations=8,
)

Production notes

• Schema drift: If you change a function signature, restart the agent. Claude receives the schema at loop start; hot-swapping tools mid-session is not supported by the current FunctionAgent.
• Token metering: Every tool result is sent back into the context window. Trim verbose returns—pass only the minimal JSON the model needs.
• Caching: When using a gateway that forwards cache-control hints, mark static system prompts with cache_control at the provider level to cut repeat token costs.

The pattern above is the smallest useful unit of agentic behavior: typed tools, a model that understands them, and a loop that reconciles the two. From here you can add retrieval tools, human-in-the-loop approval, or parallel tool dispatch without changing the core LlamaIndex wiring.