Debugging LangChain agent tool selection errors

LangChain agent tool selection errors usually surface as the agent picking the wrong tool, hallucinating tool names, or failing to invoke any tool at all. The root cause is almost always a mismatch between what the model sees (tool schemas, descriptions, few-shot examples) and what your code actually expects. This guide walks through a systematic debugging process you can run in a single notebook session.

Step 1: Reproduce with a minimal agent and verbose logging

Strip your agent down to the smallest possible configuration that still exhibits the problem. Remove memory, callbacks, custom parsers, and any tools not directly involved in the failure. Turn on verbose mode so you see every intermediate step.

from langchain.agents import initialize_agent, AgentType
from langchain.tools import BaseTool
from langchain.chat_models import ChatOpenAI
from pydantic import BaseModel, Field
from typing import Type

class DummyInput(BaseModel):
    query: str = Field(description="Search query")

class DummyTool(BaseTool):
    name = "search"
    description = "Search the web for current information"
    args_schema: Type[BaseModel] = DummyInput

    def _run(self, query: str) -> str:
        return f"Results for: {query}"

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
tools = [DummyTool()]

agent = initialize_agent(
    tools,
    llm,
    agent=AgentType.OPENAI_FUNCTIONS,
    verbose=True,
    handle_parsing_errors=True,
)

result = agent.run("What is the capital of France?")
print(result)

Run this. If the agent works here but fails in your full setup, the issue is environmental (memory, other tools, callbacks). If it fails here, you have a schema or prompting problem — continue to Step 2.

Verify success: The agent should call search with {"query": "capital of France"} and return a string containing “Results for: capital of France”.

Step 2: Inspect the function schemas sent to the model

LangChain converts each tool’s args_schema into a JSON Schema and passes it to the model via the functions parameter (OpenAI) or tools parameter (Anthropic, etc.). Print the exact schema the model receives.

from langchain.agents.openai_functions_agent.base import OpenAIFunctionsAgent
from langchain.schema import SystemMessage

# Recreate the agent to access internals
agent = initialize_agent(
    tools,
    llm,
    agent=AgentType.OPENAI_FUNCTIONS,
    verbose=False,
)

# The agent's prompt contains the function definitions
prompt = agent.agent.llm_chain.prompt
print("=== SYSTEM MESSAGE ===")
for msg in prompt.messages:
    if isinstance(msg, SystemMessage):
        print(msg.content[:2000])
        print("...")

# Or inspect the function schemas directly
for tool in tools:
    print(f"\n=== {tool.name} schema ===")
    print(tool.args_schema.schema_json(indent=2))

Look for:

• Missing description fields on schema properties — the model uses these to decide what to pass
• Incorrect type values (e.g., string vs array) — causes validation errors
• Extra properties not in your _run signature — the model may hallucinate them
• Required fields missing from required array — the model may omit them

Verify success: Every tool’s schema matches its _run/_arun signature exactly, all properties have descriptions, and required lists all non-optional parameters.

Step 3: Check tool descriptions for ambiguity

The model selects tools based entirely on the description string. Vague or overlapping descriptions cause misselection. Print each tool’s description and ask: “Would a human pick the right tool given only this sentence?”

for tool in tools:
    print(f"{tool.name}: {tool.description}")

Common problems:

• Too generic: “Search for information” — matches everything
• Overlapping: Two tools described as “Get data from the database”
• Missing intent signals: No keywords the user query would naturally contain

Rewrite descriptions to be discriminative. Include the specific trigger phrases a user would say.

class WeatherTool(BaseTool):
    name = "weather"
    description = (
        "Get current weather conditions and forecasts for a specific location. "
        "Use when the user asks about temperature, rain, snow, humidity, or weather "
        "in a city or region. Do not use for historical climate data."
    )
    # ...

Verify success: Given 5-10 sample user queries, you can predict which tool the model should pick based solely on descriptions.

Step 4: Add few-shot examples to the agent prompt

If descriptions aren’t enough, the model needs examples of correct tool selection. LangChain’s OpenAIFunctionsAgent supports few-shot prompting via the examples parameter in the prompt template.

from langchain.agents.openai_functions_agent.base import OpenAIFunctionsAgent
from langchain.prompts import MessagesPlaceholder
from langchain.schema import HumanMessage, AIMessage, FunctionMessage
import json

examples = [
    HumanMessage(content="What's the weather in Tokyo?"),
    AIMessage(
        content="",
        additional_kwargs={
            "function_call": {
                "name": "weather",
                "arguments": json.dumps({"location": "Tokyo", "unit": "celsius"})
            }
        }
    ),
    FunctionMessage(name="weather", content='{"temp": 18, "condition": "cloudy"}'),
    AIMessage(content="It's 18°C and cloudy in Tokyo."),
]

# Build a custom prompt with examples
from langchain.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant with access to tools."),
    MessagesPlaceholder(variable_name="examples", optional=True),
    ("human", "{input}"),
    MessagesPlaceholder(variable_name="agent_scratchpad"),
])

agent = OpenAIFunctionsAgent(
    llm=llm,
    tools=tools,
    prompt=prompt.partial(examples=examples),
)

from langchain.agents import AgentExecutor
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
executor.run("What's the weather in Berlin?")

Verify success: The agent selects the correct tool on queries similar to your examples, and the function call arguments match the schema.

Step 5: Validate function call arguments before execution

Even with correct tool selection, the model may pass malformed arguments (wrong types, missing required fields, extra fields). LangChain’s handle_parsing_errors=True catches some, but you should validate explicitly in your tool’s _run method.

from pydantic import ValidationError

class StrictTool(BaseTool):
    name = "strict_search"
    description = "Search with strict argument validation"
    args_schema: Type[BaseModel] = DummyInput

    def _run(self, query: str) -> str:
        # Pydantic validates on instantiation, but double-check
        try:
            validated = DummyInput(query=query)
        except ValidationError as e:
            return f"Invalid arguments: {e}"
        return f"Results for: {validated.query}"

    async def _arun(self, query: str) -> str:
        return self._run(query)

For more control, override _run to accept **kwargs and validate manually:

class FlexibleTool(BaseTool):
    name = "flexible_search"
    description = "Search with flexible input handling"
    args_schema: Type[BaseModel] = DummyInput

    def _run(self, **kwargs) -> str:
        # Handle common model mistakes
        query = kwargs.get("query") or kwargs.get("q") or kwargs.get("search_term")
        if not query:
            return "Error: missing query parameter (expected 'query', 'q', or 'search_term')"
        return f"Results for: {query}"

Verify success: Pass deliberately malformed arguments via a test harness — the tool returns a clear error message instead of crashing or producing garbage.

Step 6: Debug the agent scratchpad (intermediate steps)

The agent_scratchpad contains the history of tool calls and observations. When the agent loops or picks wrong tools, the scratchpad reveals why. Add a callback to log it.

from langchain.callbacks.base import BaseCallbackHandler
from typing import Any, Dict, List
from langchain.schema import AgentAction, AgentFinish

class ScratchpadLogger(BaseCallbackHandler):
    def on_agent_action(self, action: AgentAction, **kwargs) -> None:
        print(f"\n[TOOL CALL] {action.tool}")
        print(f"  Input: {action.tool_input}")
        print(f"  Log: {action.log}")

    def on_tool_end(self, output: str, **kwargs) -> None:
        print(f"[TOOL RESULT] {output[:200]}...")

    def on_agent_finish(self, finish: AgentFinish, **kwargs) -> None:
        print(f"\n[FINAL OUTPUT] {finish.return_values}")

agent = initialize_agent(
    tools,
    llm,
    agent=AgentType.OPENAI_FUNCTIONS,
    verbose=False,
    callbacks=[ScratchpadLogger()],
)

agent.run("What is the capital of France?")

Watch for:

• Repeated identical tool calls — the model isn’t learning from observations
• Tool calls with empty or nonsense arguments — schema or description issue
• Observations not influencing next step — prompt template may not include scratchpad correctly

Verify success: Each tool call moves the task forward; the final answer incorporates tool results.

Step 7: Test with a weaker model to expose prompt fragility

GPT-4o and Claude 3.5 Sonnet are forgiving. If your agent works only on top-tier models, your prompts are brittle. Test with a smaller model (GPT-4o-mini, Claude 3 Haiku, or a local model via Ollama) to find edge cases.

# Swap model for testing
llm_weak = ChatOpenAI(model="gpt-4o-mini", temperature=0)
# or via n4n.ai: ChatOpenAI(model="anthropic/claude-3-haiku", base_url="https://api.n4n.ai/v1", ...)

agent_weak = initialize_agent(
    tools,
    llm_weak,
    agent=AgentType.OPENAI_FUNCTIONS,
    verbose=True,
)

agent_weak.run("What is the capital of France?")

If the weaker model fails, strengthen:

• Tool descriptions (more discriminative)
• Few-shot examples (more diverse)
• System prompt (explicit reasoning instructions)

Verify success: The agent succeeds on at least two model tiers with meaningfully different capabilities.

Step 8: Handle provider-specific function calling quirks

Not all providers implement OpenAI’s function calling identically. Anthropic uses tools with a different schema format. Some local models (via Ollama, vLLM) require specific prompt templates. If you route through a gateway that normalizes this (like n4n.ai), you still need to know what the underlying model expects.

# Anthropic-style tool definition (for reference)
anthropic_tool = {
    "name": "search",
    "description": "Search the web",
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "Search query"}
        },
        "required": ["query"]
    }
}

# OpenAI-style (what LangChain generates)
openai_function = {
    "name": "search",
    "description": "Search the web",
    "parameters": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "Search query"}
        },
        "required": ["query"]
    }
}

Key differences:

• Anthropic: input_schema at top level, no parameters wrapper
• OpenAI: parameters wrapper, type: "object" required
• Some local models: require function_calling prompt template in the model card

If you switch providers and tool selection breaks, compare the actual request payloads. Log the raw HTTP request from the LLM client.

import httpx

class RequestLogger(httpx.BaseTransport):
    def __init__(self, wrapped):
        self.wrapped = wrapped

    def handle_request(self, request):
        print(f">>> {request.method} {request.url}")
        print(request.read().decode()[:2000])
        return self.wrapped.handle_request(request)

# Only for debugging — attach to your HTTP client

Verify success: The same agent configuration works across at least two providers (e.g., OpenAI and Anthropic) without code changes.

Step 9: Add structured output parsing for critical tools

For tools where argument correctness is non-negotiable (API calls, database writes), don’t rely on the model’s raw function call. Parse and validate the output through a Pydantic model after the tool returns, and feed validation errors back to the agent.

from langchain.tools import Tool
from pydantic import BaseModel, validator

class SearchResult(BaseModel):
    results: List[str]
    source: str

    @validator("results")
    def non_empty(cls, v):
        if not v:
            raise ValueError("results must not be empty")
        return v

def search_with_validation(query: str) -> str:
    raw = external_search_api(query)  # your real call
    try:
        validated = SearchResult.parse_raw(raw)
        return validated.json()
    except ValidationError as e:
        return f"VALIDATION_ERROR: {e}"

validated_tool = Tool(
    name="validated_search",
    func=search_with_validation,
    description="Search with guaranteed valid output structure",
    args_schema=DummyInput,
)

The agent sees the validation error as a tool observation and can retry with corrected arguments.

Verify success: Introduce a deliberate schema violation in the external API response — the agent retries and eventually succeeds or fails gracefully.

Step 10: Build a regression test suite

Once fixed, codify the failing cases as automated tests. Use pytest with a fixture that runs the agent against a fixed set of queries and asserts tool selection and final answer quality.

# test_agent_tool_selection.py
import pytest
from langchain.agents import AgentExecutor

TEST_CASES = [
    {
        "query": "What's the weather in Tokyo?",
        "expected_tool": "weather",
        "expected_args": {"location": "Tokyo"},
        "must_contain": ["°C", "Tokyo"],
    },
    {
        "query": "Search for LangChain tutorials",
        "expected_tool": "search",
        "expected_args": {"query": "LangChain tutorials"},
        "must_contain": ["Results for"],
    },
    {
        "query": "Calculate 15 * 23",
        "expected_tool": "calculator",
        "expected_args": {"expression": "15 * 23"},
        "must_contain": ["345"],
    },
]

@pytest.fixture
def agent_executor():
    return initialize_agent(
        tools=[weather_tool, search_tool, calculator_tool],
        llm=ChatOpenAI(model="gpt-4o-mini", temperature=0),
        agent=AgentType.OPENAI_FUNCTIONS,
        verbose=False,
    )

@pytest.mark.parametrize("case", TEST_CASES)
def test_tool_selection(agent_executor: AgentExecutor, case):
    # Capture intermediate steps
    from langchain.callbacks.base import BaseCallbackHandler
    
    class CaptureHandler(BaseCallbackHandler):
        def __init__(self):
            self.actions = []
        def on_agent_action(self, action, **kwargs):
            self.actions.append(action)
    
    handler = CaptureHandler()
    result = agent_executor.run(case["query"], callbacks=[handler])
    
    # Assert tool selection
    assert len(handler.actions) >= 1, "No tool called"
    action = handler.actions[0]
    assert action.tool == case["expected_tool"], f"Expected {case['expected_tool']}, got {action.tool}"
    
    # Assert arguments (allow extra keys)
    for k, v in case["expected_args"].items():
        assert action.tool_input.get(k) == v, f"Arg {k}: expected {v}, got {action.tool_input.get(k)}"
    
    # Assert final answer quality
    for phrase in case["must_contain"]:
        assert phrase in result, f"Result missing '{phrase}': {result}"

# Run: pytest test_agent_tool_selection.py -v

Verify success: All tests pass. Add new cases whenever you discover a new failure mode.

Summary checklist

Most tool selection errors are schema mismatches (Step 2) or ambiguous descriptions (Step 3). Start there. The remaining steps catch the long tail of prompt sensitivity, provider differences, and silent failures that only appear in production.