Connect LlamaIndex agents to external APIs

This llamaindex agents external api tutorial walks through connecting a LlamaIndex agent to live HTTP services as callable tools. You will wrap real REST endpoints in Python functions, register them with the agent runtime, and verify end-to-end execution without mocking the network.

Step 1: Install dependencies and configure the model

Start with a clean virtual environment. The only hard requirements are llama-index (which pulls llama-index-core and an LLM adapter) and an HTTP client. In this llamaindex agents external api tutorial we keep the LLM provider swappable so you can point at OpenAI, a local vLLM server, or a gateway.

pip install llama-index llama-index-llms-openai python-dotenv requests

LlamaIndex’s OpenAI class is just an OpenAI-compatible client. Set base_url to any compliant endpoint. For example, n4n.ai fronts 240+ models behind one OpenAI-compatible URL with automatic fallback when a provider is rate-limited or degraded, which removes a class of outage handling from your code.

import os
from dotenv import load_dotenv
from llama_index.llms.openai import OpenAI
from llama_index.core import Settings

load_dotenv()

Settings.llm = OpenAI(
    api_key=os.environ["LLM_API_KEY"],
    base_url=os.environ.get("LLM_BASE_URL", "https://api.openai.com/v1"),
    model=os.environ.get("LLM_MODEL", "gpt-4o-mini"),
    temperature=0,
)

Use temperature=0 for tool-calling workloads. Non-zero sampling makes the agent’s JSON arguments flaky.

Step 2: Write the external API clients

The agent will call these functions, not the HTTP layer directly. Keep them plain Python with explicit timeouts and raised exceptions. requests is synchronous; that is fine for a first cut.

import os
import requests

def get_current_weather(latitude: float, longitude: float) -> dict:
    """Fetch current temperature and precipitation from Open-Meteo."""
    url = "https://api.open-meteo.com/v1/forecast"
    params = {
        "latitude": latitude,
        "longitude": longitude,
        "current": "temperature_2m,precipitation",
    }
    resp = requests.get(url, params=params, timeout=10)
    resp.raise_for_status()
    return resp.json()["current"]

def create_github_issue(repo: str, title: str, body: str) -> dict:
    """Create an issue on a GitHub repo using a PAT from the environment."""
    token = os.environ["GITHUB_TOKEN"]
    url = f"https://api.github.com/repos/{repo}/issues"
    headers = {
        "Authorization": f"Bearer {token}",
        "Accept": "application/vnd.github+json",
    }
    resp = requests.post(
        url, json={"title": title, "body": body}, headers=headers, timeout=10
    )
    resp.raise_for_status()
    return {"html_url": resp.json()["html_url"]}

Error handling

raise_for_status() turns 4xx/5xx into exceptions. The agent runtime will surface the error as a tool failure, and a well-prompted model will retry or report. Do not swallow errors inside the function; return a dict with an error key only if you want the model to see a structured message instead of an exception trace.

Step 3: Register functions as LlamaIndex tools

LlamaIndex’s FunctionTool introspects the function signature and docstring. The description is not documentation—it is the primary signal the model uses to decide whether to call the tool. Write it like a contract.

from llama_index.core.tools import FunctionTool

weather_tool = FunctionTool.from_defaults(
    fn=get_current_weather,
    name="get_current_weather",
    description=(
        "Get current temperature (°C) and precipitation (mm) for a "
        "latitude/longitude. Inputs: latitude float, longitude float."
    ),
)

issue_tool = FunctionTool.from_defaults(
    fn=create_github_issue,
    name="create_github_issue",
    description=(
        "Create a GitHub issue. Inputs: repo string 'owner/name', "
        "title string, body string. Returns the issue HTML URL."
    ),
)

If the model passes wrong types, the agent loop will catch the TypeError and try again. Precise descriptions reduce those wasted round-trips.

Step 4: Instantiate the agent and run a task

FunctionAgent is the current generic ReAct-style runner in llama-index-core. Give it the tools and a system prompt that scopes behavior.

from llama_index.core.agent import FunctionAgent

agent = FunctionAgent(
    tools=[weather_tool, issue_tool],
    system_prompt=(
        "You are an ops assistant. Use get_current_weather to check conditions "
        "and create_github_issue to file alerts when precipitation exceeds 0."
    ),
)

response = agent.chat(
    "Check weather at 37.77,-122.42. If precipitation > 0, open an issue "
    "in 'acme/infra' titled 'Rain alert' with body 'Precip detected'."
)
print(response)

The agent will emit one or more tool calls, execute them, feed results back to the model, and return a final natural-language answer. The pattern from this llamaindex agents external api tutorial works for any REST API that can be expressed as a function.

Step 5: Manage credentials securely

Never hardcode tokens. Load them from environment variables via python-dotenv as shown, and keep .env out of version control.

echo "LLM_API_KEY=sk-..." >> .env
echo "GITHUB_TOKEN=ghp_..." >> .env
echo "LLM_BASE_URL=https://api.openai.com/v1" >> .env
echo "LLM_MODEL=gpt-4o-mini" >> .env

For GitHub, scope the PAT to repo only. For production, inject these from a secrets manager (Vault, AWS Secrets Manager) at process start rather than a flat file.

Step 6: Verify the agent actually called the APIs

Verification is non-negotiable. A green print(response) does not prove the issue was created. Add a test that patches the network boundary and asserts the tool fired with expected args, then do one real run and check the GitHub UI.

# test_agent.py
import pytest
from llama_index.core.agent import FunctionAgent
from llama_index.core.tools import FunctionTool

def test_agent_triggers_issue(monkeypatch):
    calls = {}

    def fake_weather(lat, lon):
        return {"temperature_2m": 12.0, "precipitation": 1.5}

    def fake_issue(repo, title, body):
        calls["repo"] = repo
        calls["title"] = title
        return {"html_url": "https://github.com/acme/infra/issues/1"}

    monkeypatch.setattr("__main__.get_current_weather", fake_weather)
    monkeypatch.setattr("__main__.create_github_issue", fake_issue)

    weather_tool = FunctionTool.from_defaults(fn=fake_weather, name="get_current_weather",
        description="Get weather. Inputs: latitude float, longitude float.")
    issue_tool = FunctionTool.from_defaults(fn=fake_issue, name="create_github_issue",
        description="Create issue. Inputs: repo, title, body.")

    agent = FunctionAgent(tools=[weather_tool, issue_tool],
        system_prompt="File an issue if precipitation > 0.")
    agent.chat("Weather at 37.77,-122.42; precip > 0 means alert acme/infra.")

    assert calls.get("repo") == "acme/infra"
    assert "alert" in calls.get("title", "").lower()

Run pytest test_agent.py. Then execute the real script once and confirm the issue appears in acme/infra. That two-step check closes the loop.

Step 7: Harden for production

The code above is a starting point. Real deployments need resilience and structure.

Timeouts and retries

Wrap outbound calls with tenacity. A single hung API should not block the agent indefinitely.

from tenacity import retry, stop_after_attempt(3), wait_fixed(2)

@retry(stop=stop_after_attempt(3), wait=wait_fixed(2))
def get_current_weather(latitude: float, longitude: float) -> dict:
    # ... same requests.get with timeout=5 ...

Schema validation with Pydantic

Instead of raw dicts, define input models. LlamaIndex can bind Pydantic models to tools via FunctionTool.from_defaults(fn=..., fn_schema=MyModel), which forces the model to emit valid fields.

from pydantic import BaseModel

class WeatherArgs(BaseModel):
    latitude: float
    longitude: float

Async tools

For high concurrency, define async def clients with httpx.AsyncClient and pass async_fn= to FunctionTool.from_defaults. The agent runner supports async natively; you just avoid blocking the event loop.

Observability

Log every tool invocation with its arguments and latency. The agent’s intermediate steps are accessible via response.metadata on some providers, but a simple decorator on your functions is more reliable. Per-token usage metering (available on gateways like n4n.ai) lets you attribute cost to specific agent runs if you forward the same user or session header.

Building LlamaIndex agents that talk to external APIs is mostly disciplined function wrapping. The framework stays out of your I/O path, which is the correct design. Write tight tools, verify the network calls, and add retries only at the boundaries.