LangChain tool calling with Llama 3.3 70B on n4n.ai

Llama 3.3 70B supports native tool calling, and LangChain’s ChatOpenAI class works out of the box with any OpenAI-compatible endpoint. This guide walks through wiring up langchain llama 3.3 70b tool calling n4n.ai from a fresh virtual environment to a verified agent that executes Python functions. You’ll end up with a minimal, reproducible pattern you can drop into a larger system.

Step 1: Create the environment and install dependencies

Start with a clean Python 3.11+ environment. Pin versions to avoid surprise breakage when LangChain or its OpenAI integration updates.

python -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install "langchain==0.3.7" "langchain-openai==0.2.1" "langchain-core==0.3.6" "python-dotenv==1.0.1"

Create a .env file in the project root. The only required secret is your n4n.ai API key. The base URL points at the n4n.ai gateway; the model identifier is the provider-scoped name Llama 3.3 70B uses there.

# .env
N4N_API_KEY=sk-your-key-here
N4N_BASE_URL=https://api.n4n.ai/v1
MODEL_NAME=meta-llama/llama-3.3-70b-instruct

Step 2: Define the tools your agent can call

LangChain tools are plain Python functions decorated with @tool. The docstring becomes the function schema the model sees, so write it like you’d document a public API — parameter types, units, and constraints all matter.

# tools.py
from langchain_core.tools import tool
from typing import Annotated
import httpx
import json

@tool
def get_weather(
    latitude: Annotated[float, "Latitude in decimal degrees"],
    longitude: Annotated[float, "Longitude in decimal degrees"],
) -> str:
    """Fetch current weather for a coordinate pair using Open-Meteo."""
    url = "https://api.open-meteo.com/v1/forecast"
    params = {
        "latitude": latitude,
        "longitude": longitude,
        "current_weather": "true",
        "timezone": "auto",
    }
    resp = httpx.get(url, params=params, timeout=10.0)
    resp.raise_for_status()
    data = resp.json()["current_weather"]
    return json.dumps({
        "temperature_c": data["temperature"],
        "windspeed_kmh": data["windspeed"],
        "weather_code": data["weathercode"],
        "observed_at": data["time"],
    })

@tool
def calculate_compound_interest(
    principal: Annotated[float, "Starting amount"],
    annual_rate: Annotated[float, "Annual rate as a decimal, e.g. 0.07 for 7%"],
    years: Annotated[int, "Number of years"],
    compounds_per_year: Annotated[int, "Compounding frequency per year"] = 12,
) -> str:
    """Return future value of an investment with compound interest."""
    amount = principal * (1 + annual_rate / compounds_per_year) ** (compounds_per_year * years)
    return json.dumps({
        "future_value": round(amount, 2),
        "principal": principal,
        "annual_rate": annual_rate,
        "years": years,
    })

The return type is str containing JSON because the current tool-calling pathway in LangChain expects string content. If you prefer structured output, wrap the tool with StructuredTool.from_function and set return_direct=False; the string approach keeps the example minimal.

Step 3: Configure the ChatOpenAI client for n4n.ai

ChatOpenAI accepts base_url and api_key parameters, so pointing it at n4n.ai is a one-liner. Set model to the identifier from .env. Temperature zero makes tool selection deterministic; adjust if you want more creative routing.

# client.py
import os
from langchain_openai import ChatOpenAI
from dotenv import load_dotenv

load_dotenv()

def make_llm() -> ChatOpenAI:
    return ChatOpenAI(
        model=os.getenv("MODEL_NAME", "meta-llama/llama-3.3-70b-instruct"),
        api_key=os.getenv("N4N_API_KEY"),
        base_url=os.getenv("N4N_BASE_URL"),
        temperature=0,
        max_tokens=4096,
        timeout=60,
        max_retries=2,
    )

The gateway honors standard OpenAI parameters. If you need to steer routing — for example, prefer a specific provider or enable caching — pass extra headers via default_headers or use the extra_body parameter on individual calls. The n4n.ai gateway forwards provider cache-control hints automatically, so repeated identical tool calls can hit a cached response without extra code.

Step 4: Build the agent graph with LangGraph

LangGraph replaced the legacy AgentExecutor as the recommended way to run tool-calling agents. The prebuilt create_react_agent compiles a graph that loops: model → tool → model → … until the model returns a final answer without a tool call.

# agent.py
from langgraph.prebuilt import create_react_agent
from langchain_core.messages import HumanMessage, SystemMessage
from client import make_llm
from tools import get_weather, calculate_compound_interest

SYSTEM_PROMPT = """You are a helpful assistant with access to tools.
Use get_weather for current conditions at coordinates.
Use calculate_compound_interest for financial projections.
Always call tools when users ask for data you don't know.
Return final answers in plain language, not JSON."""

def build_agent():
    llm = make_llm()
    tools = [get_weather, calculate_compound_interest]
    return create_react_agent(llm, tools, state_modifier=SYSTEM_PROMPT)

state_modifier injects the system prompt into every turn. You can also pass a callable that receives the current state and returns a list of messages — useful for dynamic instructions or few-shot examples.

Step 5: Run a verification script

Create a script that exercises both tools in a single conversation. This proves the model selects the right tool, passes valid arguments, and synthesizes the results.

# verify.py
import asyncio
from agent import build_agent
from langchain_core.messages import HumanMessage

async def main():
    agent = build_agent()
    
    # Turn 1: weather lookup
    result = await agent.ainvoke({
        "messages": [HumanMessage(content="What's the weather in San Francisco right now?")]
    })
    print("=== Turn 1 ===")
    for msg in result["messages"]:
        print(f"{msg.type}: {msg.content[:200]}")
    
    # Turn 2: compound interest
    result = await agent.ainvoke({
        "messages": [
            HumanMessage(content="If I invest $10,000 at 7% annually, compounded monthly, what will I have in 30 years?")
        ]
    })
    print("\n=== Turn 2 ===")
    for msg in result["messages"]:
        print(f"{msg.type}: {msg.content[:200]}")

if __name__ == "__main__":
    asyncio.run(main())

Run it:

python verify.py

Expected output structure

You should see three message types per turn:

1. HumanMessage — your prompt
2. AIMessage with tool_calls — the model’s decision, including function name and arguments
3. ToolMessage — the tool’s JSON string return value
4. AIMessage (final) — natural-language answer incorporating the tool result

If the final answer references the temperature, wind speed, or the computed future value (~$76,122), the pipeline works end to end.

Step 6: Inspect raw requests for debugging

When tool calling fails — wrong arguments, missing calls, hallucinated functions — you need to see what the model actually received and sent. LangChain’s callbacks let you capture the full request/response cycle.

# debug.py
from langchain_core.callbacks import BaseCallbackHandler
from typing import Any, Dict, List
import json

class PrintCallback(BaseCallbackHandler):
    def on_llm_start(self, serialized: Dict[str, Any], prompts: List[str], **kwargs):
        print("=== PROMPT TO MODEL ===")
        for p in prompts:
            print(p[:2000])
    
    def on_llm_end(self, response, **kwargs):
        print("\n=== RAW MODEL RESPONSE ===")
        print(response.generations[0][0].text[:2000])
    
    def on_tool_start(self, serialized: Dict[str, Any], input_str: str, **kwargs):
        print(f"\n=== TOOL START: {serialized.get('name')} ===")
        print(input_str)
    
    def on_tool_end(self, output: str, **kwargs):
        print(f"\n=== TOOL OUTPUT ===")
        print(output[:1000])

# Usage: pass callbacks=[PrintCallback()] to ainvoke()

Attach it to a single invocation:

result = await agent.ainvoke(
    {"messages": [HumanMessage(content="Weather in Tokyo?")]},
    config={"callbacks": [PrintCallback()]}
)

This prints the serialized prompt (including the injected tool schemas), the model’s raw tool-call JSON, and the tool’s return value. Compare the schema in the prompt against your @tool docstrings — mismatches are the most common cause of invalid_arguments errors.

Step 7: Handle streaming and token usage

Production agents stream tokens to the UI. LangGraph’s astream yields state updates per node. The final chunk contains the complete message list; intermediate chunks show tool calls as they happen.

# stream_example.py
async def stream_demo():
    agent = build_agent()
    async for chunk in agent.astream(
        {"messages": [HumanMessage(content="Weather in London?")]},
        stream_mode="values",
    ):
        # chunk is the full state dict at each step
        last_msg = chunk["messages"][-1]
        print(f"{last_msg.type}: {getattr(last_msg, 'content', '')[:120]}")

For per-token usage metering, the n4n.ai gateway returns standard OpenAI usage fields in the final AIMessage. Access them via response.usage_metadata (LangChain 0.3+) or response.response_metadata["token_usage"] on the raw chunk.

final_msg = result["messages"][-1]
if hasattr(final_msg, "usage_metadata"):
    print(f"Prompt tokens: {final_msg.usage_metadata['input_tokens']}")
    print(f"Completion tokens: {final_msg.usage_metadata['output_tokens']}")

Step 8: Add structured output for the final answer

If downstream systems need typed responses, wrap the agent with a structured-output parser. Define a Pydantic model for the final answer, then use with_structured_output on the LLM before passing it to create_react_agent.

# structured_agent.py
from pydantic import BaseModel, Field
from langgraph.prebuilt import create_react_agent
from client import make_llm
from tools import get_weather, calculate_compound_interest

class WeatherAnswer(BaseModel):
    location: str = Field(description="City or coordinates queried")
    temperature_c: float
    conditions: str = Field(description="Human-readable weather description")

class InvestmentAnswer(BaseModel):
    principal: float
    annual_rate: float
    years: int
    future_value: float
    monthly_contribution: float = 0

# Note: with_structured_output only constrains the FINAL answer,
# not intermediate tool calls. The model still calls tools normally.
llm = make_llm().with_structured_output(WeatherAnswer | InvestmentAnswer)

agent = create_react_agent(
    llm,
    [get_weather, calculate_compound_interest],
    state_modifier=SYSTEM_PROMPT,
)

The union type lets the model pick the right schema based on the query. The final AIMessage will have .content as a parsed Pydantic instance instead of a string.

Common failure modes and fixes

What to take forward

You now have a working langchain llama 3.3 70b tool calling n4n.ai setup that:

• Uses ChatOpenAI with a custom base URL — no vendor-specific SDK
• Defines tools as typed Python functions with schema-generating docstrings
• Runs via LangGraph’s create_react_agent for correct loop semantics
• Streams tokens and exposes usage metadata for observability
• Supports structured final output without constraining tool calls

From here, add persistent checkpointers (MemorySaver, PostgresSaver) for multi-turn conversations, attach a ToolNode with custom error handling for retries, or swap the model identifier to any of the 240+ models the gateway serves without changing application code. The pattern stays the same.