LangChain function calling on Claude 3.5 Sonnet

LangChain function calling with Claude 3.5 Sonnet works differently than OpenAI models. Anthropic’s tool use API expects a specific message format, and LangChain’s abstraction layer handles most of the translation — but only if you wire it correctly. This guide walks through the complete setup, from installing the right packages to debugging why your tools aren’t firing.

Step 1: install the correct dependencies

You need three packages minimum. The Anthropic provider lives in langchain-anthropic, not the main LangChain package.

pip install langchain-anthropic langchain-core langchain

If you’re using LangGraph for agent orchestration (recommended), add:

pip install langgraph

Verify versions at minimum:

• langchain-anthropic>=0.1.0
• langchain-core>=0.2.0
• langchain>=0.2.0

Older versions lack the bind_tools method that makes Claude tool calling ergonomic.

Step 2: define your tools as Pydantic models

Claude’s tool schema validation is strict. Use Pydantic v2 models with Field descriptions — these become the schema sent to the model.

from pydantic import BaseModel, Field
from typing import Literal


class GetWeather(BaseModel):
    """Get current weather for a location."""
    location: str = Field(description="City and state, e.g. 'San Francisco, CA'")
    unit: Literal["celsius", "fahrenheit"] = Field(
        default="fahrenheit",
        description="Temperature unit"
    )


class SearchWeb(BaseModel):
    """Search the web for recent information."""
    query: str = Field(description="Search query")
    max_results: int = Field(default=5, ge=1, le=10)

The docstrings and field descriptions matter. Claude uses them to decide which tool to invoke and how to populate arguments.

Step 3: initialize the model with bound tools

This is where most tutorials go wrong. You must use bind_tools, not the legacy functions parameter.

from langchain_anthropic import ChatAnthropic
from langchain_core.messages import HumanMessage

llm = ChatAnthropic(
    model="claude-3-5-sonnet-20241022",
    temperature=0,
    max_tokens=4096,
).bind_tools([GetWeather, SearchWeb])

Key parameters:

• temperature=0 — deterministic tool selection
• max_tokens=4096 — leave room for tool calls and responses
• Model string must match exactly; claude-3-5-sonnet-latest also works but pins to a moving target

Test the binding directly:

response = llm.invoke([HumanMessage(content="What's the weather in Tokyo?")])
print(response.tool_calls)

You should see a tool_calls list with name: "GetWeather" and parsed arguments. If tool_calls is empty, the model didn’t recognize the intent — check your descriptions.

Step 4: build the tool execution layer

LangChain doesn’t execute tools automatically. You need a function that maps tool calls to actual implementations.

import json
from typing import Any
from langchain_core.messages import ToolMessage


def execute_tool(tool_call: dict[str, Any]) -> ToolMessage:
    """Execute a single tool call and return a ToolMessage."""
    name = tool_call["name"]
    args = tool_call["args"]
    tool_call_id = tool_call["id"]

    if name == "GetWeather":
        # Replace with real API call
        result = {"temperature": 72, "condition": "sunny", "location": args["location"]}
    elif name == "SearchWeb":
        # Replace with real search API
        result = [{"title": "Result 1", "url": "https://example.com"}]
    else:
        result = {"error": f"Unknown tool: {name}"}

    return ToolMessage(
        content=json.dumps(result),
        tool_call_id=tool_call_id,
        name=name,
    )

The ToolMessage must include the original tool_call_id so Claude can correlate the response with its request.

Step 5: wire the loop with LangGraph

LangGraph handles the agent loop — model calls tools, tools return results, model decides next step. This is the production pattern.

from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from typing import TypedDict, Annotated
from langchain_core.messages import BaseMessage
import operator


class AgentState(TypedDict):
    messages: Annotated[list[BaseMessage], operator.add]


def call_model(state: AgentState):
    response = llm.invoke(state["messages"])
    return {"messages": [response]}


def should_continue(state: AgentState) -> Literal["tools", "end"]:
    last_message = state["messages"][-1]
    if last_message.tool_calls:
        return "tools"
    return "end"


tool_node = ToolNode([GetWeather, SearchWeb])

workflow = StateGraph(AgentState)
workflow.add_node("agent", call_model)
workflow.add_node("tools", tool_node)
workflow.set_entry_point("agent")
workflow.add_conditional_edges("agent", should_continue)
workflow.add_edge("tools", "agent")

app = workflow.compile()

The ToolNode from langgraph.prebuilt handles the execution loop for you. It expects tools as LangChain BaseTool instances, not raw Pydantic models. Convert them:

from langchain_core.tools import StructuredTool

get_weather_tool = StructuredTool.from_function(
    func=lambda location, unit: {"temperature": 72, "condition": "sunny"},
    name="GetWeather",
    description="Get current weather for a location.",
    args_schema=GetWeather,
)

search_web_tool = StructuredTool.from_function(
    func=lambda query, max_results: [{"title": "Result 1", "url": "https://example.com"}],
    name="SearchWeb",
    description="Search the web for recent information.",
    args_schema=SearchWeb,
)

tool_node = ToolNode([get_weather_tool, search_web_tool])

Step 6: run the agent end-to-end

from langchain_core.messages import HumanMessage

inputs = {"messages": [HumanMessage(content="What's the weather in Tokyo and Seattle?")]}

for chunk in app.stream(inputs, stream_mode="values"):
    message = chunk["messages"][-1]
    if hasattr(message, "tool_calls") and message.tool_calls:
        print(f"Tool calls: {message.tool_calls}")
    elif isinstance(message, ToolMessage):
        print(f"Tool result: {message.content}")
    else:
        print(f"Assistant: {message.content}")

Expected output sequence:

1. Assistant message with tool_calls for both locations
2. Two ToolMessage entries with weather data
3. Final assistant message synthesizing the results

Step 7: handle Claude-specific quirks

Tool choice forcing

Claude sometimes refuses to call tools even when appropriate. Force it:

llm_forced = llm.bind(tool_choice={"type": "tool", "name": "GetWeather"})

Or for any tool:

llm_any = llm.bind(tool_choice="any")

Parallel tool calls

Claude 3.5 Sonnet supports parallel tool invocation natively. The tool_calls list will contain multiple entries. Your execution layer must handle them all — the ToolNode does this automatically.

Token accounting

Tool definitions consume context. Each tool schema adds ~200-500 tokens. With 10+ tools, you’re burning 2-5k tokens before the first user message. Keep tool schemas minimal; omit optional fields you don’t need.

System prompts

Inject a system message to steer tool usage:

from langchain_core.messages import SystemMessage

system = SystemMessage(content="""
You have access to weather and search tools. 
Always use GetWeather for current conditions.
Use SearchWeb for facts that change frequently.
Never hallucinate tool results.
""")

inputs = {"messages": [system, HumanMessage(content="Weather in Tokyo?")]}

Step 8: streaming for production UX

Blocking on the full agent loop feels slow. Stream the model output and tool events separately.

async def stream_agent(query: str):
    inputs = {"messages": [HumanMessage(content=query)]}
    
    async for event in app.astream_events(inputs, version="v2"):
        kind = event["event"]
        if kind == "on_chat_model_stream":
            content = event["data"]["chunk"].content
            if content:
                yield {"type": "token", "content": content}
        elif kind == "on_tool_start":
            yield {"type": "tool_start", "name": event["name"], "input": event["data"]["input"]}
        elif kind == "on_tool_end":
            yield {"type": "tool_end", "name": event["name"], "output": event["data"]["output"]}

Consume this in your frontend to show typing indicators, tool spinners, and progressive results.

Step 9: error handling and retries

Tools fail. Network errors, rate limits, bad inputs. Wrap execution:

from tenacity import retry, stop_after_attempt, wait_exponential


@retry(
    wait=wait_exponential(multiplier=1, min=2, max=10),
    stop=stop_after_attempt(3),
)
async def safe_execute(tool_call: dict) -> ToolMessage:
    try:
        return execute_tool(tool_call)
    except Exception as e:
        return ToolMessage(
            content=json.dumps({"error": str(e)}),
            tool_call_id=tool_call["id"],
            name=tool_call["name"],
        )

The ToolNode accepts a handle_tool_errors parameter for simpler cases:

tool_node = ToolNode(
    [get_weather_tool, search_web_tool],
    handle_tool_errors=True,  # returns error as ToolMessage instead of raising
)

Step 10: verify success with a test harness

Write a test that exercises the full loop:

import pytest
from langchain_core.messages import HumanMessage, AIMessage, ToolMessage


def test_weather_agent():
    inputs = {"messages": [HumanMessage(content="Weather in Boston?")]}
    result = app.invoke(inputs)
    
    messages = result["messages"]
    
    # First message should be AI with tool_calls
    assert isinstance(messages[1], AIMessage)
    assert len(messages[1].tool_calls) == 1
    assert messages[1].tool_calls[0]["name"] == "GetWeather"
    assert messages[1].tool_calls[0]["args"]["location"] == "Boston, MA"
    
    # Second message should be ToolMessage
    assert isinstance(messages[2], ToolMessage)
    assert messages[2].name == "GetWeather"
    tool_result = json.loads(messages[2].content)
    assert "temperature" in tool_result
    
    # Final message should be AI without tool_calls
    assert isinstance(messages[3], AIMessage)
    assert not messages[3].tool_calls
    assert "boston" in messages[3].content.lower()

Run with pytest -v. This catches regressions when you upgrade LangChain or change prompts.

Common failure modes

Production checklist

Before deploying:

• Tool schemas documented and versioned
• Retry logic on all external API calls
• Token usage logging per request
• Latency budgets for tool execution (<2s p95)
• Fallback behavior when tools unavailable
• Evaluation set with expected tool call sequences

The n4n.ai gateway can simplify the model side — one endpoint addresses 240+ models including Claude 3.5 Sonnet, with automatic fallback when a provider degrades — but the LangChain integration patterns above remain the same regardless of how you reach the model.

Next steps

• Add structured output parsing with with_structured_output for the final answer
• Implement human-in-the-loop approval for destructive tools
• Build a tool registry service for dynamic tool discovery
• Set up LangSmith tracing to debug production agent runs

The pattern scales. Same graph structure works for 5 tools or 50.