LangChain create_tool_calling_agent explained

create_tool_calling_agent is LangChain’s factory function that builds a runnable agent chain from a language model, a list of tools, and a prompt template. It handles the mechanics of formatting tool schemas, injecting them into the prompt, parsing model output for tool calls, executing those tools, and feeding results back to the model — all in a single Runnable you can invoke, stream, or batch. The function exists because manually stitching together prompt templates, output parsers, and tool executors is error-prone and repetitive.

What the function actually does

Under the hood, create_tool_calling_agent assembles three components you would otherwise wire yourself:

1. Prompt construction — It takes your prompt template (which must include a MessagesPlaceholder named agent_scratchpad) and binds the tool schemas into the system message using the model’s native function-calling format. For OpenAI-compatible models this means a tools array in the request; for Anthropic it means tool_use blocks; for others it falls back to a structured prompt.

2. Output parsing — It wraps the model with an OpenAIToolsAgentOutputParser (or the appropriate parser for the model family) that extracts tool_calls from the assistant message and converts them into AgentActionMessageLog objects. If the model returns plain text instead of a tool call, the parser treats it as the final answer.

3. Runnable sequence — It returns a RunnableSequence equivalent to: prompt | model.bind_tools(tools) | output_parser. You can invoke it directly, stream tokens, or plug it into AgentExecutor for multi-step loops.

The signature is minimal:

from langchain.agents import create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.tools import BaseTool

agent = create_tool_calling_agent(
    llm=model,
    tools=[search, calculator, weather],
    prompt=prompt_template,
)

The prompt must contain a MessagesPlaceholder(variable_name="agent_scratchpad"). That placeholder is where the agent writes intermediate tool calls and observations during execution. Without it, the agent cannot see its own history and will hallucinate or loop.

Why it matters for production systems

Before create_tool_calling_agent stabilized (LangChain 0.1.x), teams built agents by hand: formatting tool JSON schemas into system prompts, writing custom regex parsers for model output, handling malformed tool calls, and managing the scratchpad manually. That code was brittle — a model upgrade that changed tool-call formatting broke the parser, and every new model family required a new parser implementation.

The factory function centralizes that logic. When LangChain adds support for a new provider’s tool-calling format (say, Mistral’s function calling or Google’s function declarations), the parser updates in one place. Your agent code stays unchanged.

It also enables composability. Because the return value is a standard Runnable, you can:

• Wrap it with RunnableWithMessageHistory for conversation memory
• Pipe it through RunnablePassthrough.assign() to inject runtime context
• Use agent.with_config({"run_name": "my-agent"}) for tracing
• Swap the underlying model without rewriting the agent logic

For teams running multiple models across providers — something we see constantly at n4n.ai — this abstraction pays off immediately. The same agent definition works against GPT-4o, Claude 3.5 Sonnet, or an open-weight model served via vLLM, provided the model supports tool calling.

Concrete example: a research agent with streaming

Here is a complete, runnable example. It uses AgentExecutor to manage the multi-step loop, streams tokens to the console, and includes a minimal prompt that satisfies the scratchpad requirement.

import asyncio
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

# Tools
@tool
def search_web(query: str) -> str:
    """Search the web for current information."""
    # In production, call a real search API (SerpAPI, Tavily, etc.)
    return f"Search results for: {query}"

@tool
def calculate(expression: str) -> str:
    """Evaluate a mathematical expression."""
    try:
        return str(eval(expression, {"__builtins__": {}}))
    except Exception as e:
        return f"Error: {e}"

# Prompt — note the agent_scratchpad placeholder
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a research assistant. Use tools to answer questions accurately."),
    MessagesPlaceholder(variable_name="chat_history", optional=True),
    ("human", "{input}"),
    MessagesPlaceholder(variable_name="agent_scratchpad"),
])

# Model + agent + executor
model = ChatOpenAI(model="gpt-4o-mini", temperature=0)
tools = [search_web, calculate]
agent = create_tool_calling_agent(model, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

async def main():
    # Streaming: yields dicts with "output" key at the end
    async for event in executor.astream_events(
        {"input": "What's 15% of 2,400? Then search for 'compound interest formula'."},
        version="v2",
    ):
        if event["event"] == "on_chat_model_stream":
            print(event["data"]["chunk"].content, end="", flush=True)
        elif event["event"] == "on_tool_end":
            print(f"\n[Tool {event['name']} returned: {event['data']['output']}]")

asyncio.run(main())

Key details worth noticing:

• The prompt includes MessagesPlaceholder(variable_name="chat_history", optional=True) so you can pass conversation history via RunnableWithMessageHistory later.
• AgentExecutor handles the loop: it invokes the agent, executes any tool calls, feeds observations back, and repeats until the model returns a final answer.
• astream_events (v2) gives you granular streaming — model tokens, tool start/end, and final output — without buffering the whole response.
• verbose=True logs each step to stdout; in production you’d route those logs to your observability stack.

Common misconceptions

“It works with any model”

False. create_tool_calling_agent requires a model that supports native tool/function calling — meaning the model emits structured tool_calls in its response, not plain text that you parse with regex. Models without native support (older GPT-3.5 variants, many open-weight models served without a function-calling adapter) will not work. For those, you need create_react_agent or create_structured_chat_agent, which use prompt-based tool calling and a different parser.

If you’re routing across providers, verify each model’s capability. A model that claims “function calling” but returns malformed JSON will break the parser silently — the agent will treat the malformed output as a final answer.

“The prompt template is optional”

It is not optional. You must pass a ChatPromptTemplate with an agent_scratchpad placeholder. The function does not provide a default prompt. A common mistake is passing a string or a PromptTemplate (the old string-based class) instead of a ChatPromptTemplate — the type annotation accepts BasePromptTemplate, but the implementation expects chat messages.

“AgentExecutor and the agent are the same thing”

They are distinct. create_tool_calling_agent returns a single-step runnable: given inputs, it produces either a tool call or a final answer. AgentExecutor wraps that runnable in a loop, executes tools, manages the scratchpad, and enforces limits like max_iterations and max_execution_time. You can use the agent directly for single-turn tool use (e.g., a classifier that calls one tool and returns), but most conversational agents need the executor.

“Tool schemas are inferred perfectly from docstrings”

LangChain uses Pydantic to generate JSON schemas from tool function signatures and docstrings. This works well for simple types (str, int, float, bool, List[str]). It struggles with:

• Union types (str | int) — the schema may allow both, confusing the model
• Nested Pydantic models — sometimes flattened incorrectly
• Optional fields with defaults — may appear as required in the schema

Inspect the generated schema with tool.args_schema.schema() and override args_schema with a hand-written Pydantic model if the model consistently mis-calls the tool.

“Streaming works out of the box with no changes”

Streaming tokens works automatically because the underlying model streams. Streaming tool calls does not — the model buffers the entire tool call before emitting it. You will see a pause, then the full tool call appears at once. If you need progressive tool-call streaming (e.g., to show a search query being typed), you need a model and provider that support it (currently rare) and a custom parser. For most cases, accept that tool calls arrive atomically.

Debugging checklist

When the agent misbehaves, check these in order:

1. Scratchpad present? — Print prompt.format_messages(input="test", agent_scratchpad=[]) and verify the placeholder renders as an empty list slot.
2. Tool schemas bound? — Inspect model.bind_tools(tools).kwargs.get("tools") (OpenAI) or the equivalent for your provider. Empty list means the model doesn’t support binding or the tools lack schemas.
3. Parser matches model? — If you swapped models, the parser may be wrong. create_tool_calling_agent picks a parser based on model.__class__.__name__ heuristics. For custom wrappers, pass output_parser explicitly (undocumented but supported).
4. Max iterations? — Default AgentExecutor(max_iterations=15). Complex tasks hit this. Increase it or add a “summarize progress” tool.
5. Error handling? — Tool exceptions become observations. If a tool raises, the agent sees the traceback as a string. Wrap tools in try/except and return structured error messages the model can act on.

When to reach for something else

• No native tool calling → create_react_agent (prompt-based, works on any instruction-tuned model)
• Structured output without tools → create_json_agent or create_openapi_agent
• Multi-agent orchestration → LangGraph (state machines, not linear loops)
• Deterministic workflows → Skip agents entirely; use RunnableSequence with explicit branches

create_tool_calling_agent is the right tool when you have a tool-calling model, need multi-step reasoning with external APIs, and want the standard loop behavior with minimal boilerplate. It is not a silver bullet for every “agentic” pattern — but for the common case, it replaces a few hundred lines of fragile glue code with a single function call.