Best framework for agents that call external APIs and tools

Choosing the best framework for tool-calling agents is less about hype and more about how cleanly you can map a model’s function requests to real HTTP endpoints, SDKs, and internal services. The right choice reduces boilerplate, handles retries, and lets you swap LLM providers without rewriting your tool layer.

Below are the frameworks that consistently ship in production systems where external API calls are the core workload. Each handles the observe-decide-act loop differently; your job is to match that loop to your latency and compliance requirements.

LangChain

LangChain remains the default for many teams because its Tool abstraction and AgentExecutor handle the orchestration loop out of the box. You define a tool as a function with a schema, and the framework feeds it to the model, parses the response, and executes the call. For REST APIs, you typically wrap requests or use StructuredTool. The ecosystem includes prebuilt connectors for Stripe, Twilio, and most CRUD services.

from langchain.tools import StructuredTool
from pydantic import BaseModel

class WeatherArgs(BaseModel):
    lat: float
    lon: float

def get_weather(lat: float, lon: float) -> str:
    # call external API with timeout and retry
    return f"weather at {lat},{lon}"

weather_tool = StructuredTool.from_function(
    func=get_weather,
    name="get_weather",
    description="Fetch current weather for coordinates",
    args_schema=WeatherArgs
)

When evaluating the best framework for tool-calling agents, LangChain’s maturity is a double-edged sword. The abstraction leaks when you need fine-grained control over retries, auth headers, or streaming. You can subclass BaseTool to add async def _arun for non-blocking I/O, but the docs scatter this across versions. If your agent primarily calls ten or more external services with varying auth schemes, budget time for fighting defaults.

A practical pattern is to keep your HTTP client (httpx) outside the tool and inject it, so LangChain only serializes the result. That avoids recreating connection pools on every agent step.

LlamaIndex

LlamaIndex started as a data framework but its FunctionAgent and ToolSpec are competent for API-heavy agents. The differentiator is native support for OpenAPI specs: you can ingest a Swagger doc and generate tools automatically. This is a massive win when integrating with a partner’s API surface of dozens of endpoints.

from llama_index.core.tools import FunctionTool

def search_flights(origin: str, dest: str) -> dict:
    # call flight API
    return {"flights": []}

flight_tool = FunctionTool.from_defaults(
    fn=search_flights,
    name="search_flights",
    description="Search flights between airports"
)

For auto-generation, OpenAPISpec parses the schema and yields a ToolSpec where each operation becomes a callable. LlamaIndex’s agent loop is lighter than LangChain’s, but you lose some built-in memory management. For pure tool-calling against external APIs, that leanness is often preferable. It does not try to be everything; you compose it with your own httpx client and handle 429s yourself.

Async is first-class: async def fn works with FunctionTool.from_defaults, and the agent will await it. This matters when you fan out to multiple APIs in one reasoning step.

Microsoft AutoGen

AutoGen takes a conversational multi-agent approach. Tools are registered as functions on an AssistantAgent, and a UserProxyAgent executes them. This is powerful when external API calls need human-in-the-loop approval or when multiple agents negotiate which tool to call. The framework ships a CodeExecutor that can run generated Python, but for strict API calls you disable that and use the function map.

from autogen import AssistantAgent, UserProxyAgent, register_function

def charge_card(amount: float) -> str:
    # call payment API
    return "charged"

assistant = AssistantAgent("assistant", llm_config={"model": "gpt-4o"})
proxy = UserProxyAgent("proxy", human_input_mode="NEVER")
register_function(charge_card, caller=assistant, executor=proxy)

AutoGen’s event loop is heavier than necessary for a simple agent that hits a few CRUD endpoints. Its strength is group chat: you can have a SecurityAgent that intercepts tool calls before the ExecutorAgent fires the HTTP request. That pattern is valuable when calling external billing or PII endpoints. The cost is complexity in debugging message histories.

CrewAI

CrewAI frames tools as classes with a run method, and agents are assigned roles. It’s opinionated about process: you define a crew, give each agent a goal, and the framework sequences the work. Tool integration is straightforward, but the value is in orchestrating multiple specialized agents that each own a slice of your API surface—one calls Salesforce, another calls Slack.

from crewai import Agent, Tool

def slack_notify(message: str) -> str:
    # call Slack API
    return "sent"

notify_tool = Tool(func=slack_notify, name="Slack", description="Send Slack message")
agent = Agent(role="Notifier", goal="Alert team", tools=[notify_tool])

CrewAI hides a lot of the low-level loop, which is good for prototyping. In production, inspect how it serializes tool output into the next agent’s context—long API responses can blow up token counts fast. Use output_json or manual truncation in your tool wrapper. The framework supports async tools via async_run, but most examples are sync.

Pydantic AI

Pydantic AI is the newest contender and the most type-safe. Tools are just typed Python functions; the framework uses Pydantic models for validation and generates the JSON schema for the model automatically. If your external APIs have strict input contracts, this eliminates a whole class of runtime errors before the HTTP call is made.

from pydantic_ai import Agent, Tool
from pydantic import BaseModel

class ChargeReq(BaseModel):
    user_id: int
    cents: int

def charge(req: ChargeReq) -> str:
    # call billing API
    return "ok"

agent = Agent("openai:gpt-4o", tools=[Tool(charge)])

Because it builds on Pydantic, you get validation, mocking, and testability for free. Dependency injection lets you pass an httpx.AsyncClient to the tool at run time, keeping the LLM layer decoupled from transport. The trade-off is a smaller ecosystem and fewer prebuilt integrations than LangChain. For teams already using FastAPI and Pydantic everywhere, it’s the most natural fit and arguably the best framework for tool-calling agents that must obey strict schemas.

Production considerations

Regardless of which library you pick, the failure modes are similar: token limits from verbose API JSON, 429s from upstream, and schema drift when the external API changes. Wrap every tool in a circuit breaker. Use structured logging to capture the exact arguments the model emitted versus what the API received.

If you want to avoid vendor lock-in, point these frameworks at an OpenAI-compatible gateway such as n4n.ai that automatically falls back when a provider is rate-limited and meters per-token usage. The frameworks above all accept a base URL and API key, so the swap is a one-line config change.

Synthesis

No single winner exists; the best framework for tool-calling agents depends on your topology. If you need auto-generated tools from OpenAPI, LlamaIndex wins. If you need multi-agent oversight, AutoGen or CrewAI. For type-safe simplicity, Pydantic AI. For maximum community support, LangChain.

Pick the framework that matches your team’s existing stack, then invest the saved time in hardening the API client layer: retries, timeouts, and schema validation will break your agent long before the orchestration framework does.