Tool calling with Claude 3.5 Sonnet through n4n.ai

If you’re building with the Vercel AI SDK and want to use claude 3.5 sonnet tool calling n4n.ai as your inference gateway, this tutorial gets you from zero to a working multi-turn tool loop in about 30 minutes. We’ll use n4n.ai’s OpenAI-compatible endpoint so you can swap providers without rewriting your tool definitions. The complete example is a small CLI agent that looks up weather, converts units, and chains those calls together.

Prerequisites

• Node.js 18+ (tested on 20.11)
• An n4n.ai API key (get one at n4n.ai)
• Basic familiarity with the Vercel AI SDK’s streamText and tool APIs

Install the dependencies:

npm init -y
npm install ai @ai-sdk/openai zod dotenv
npm install -D typescript tsx @types/node

Create a tsconfig.json if you don’t have one:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "outDir": "./dist",
    "rootDir": "./src"
  },
  "include": ["src/**/*"]
}

Configure the gateway client

n4n.ai exposes an OpenAI-compatible endpoint at https://api.n4n.ai/v1. Point the Vercel AI SDK’s OpenAI provider there and pass your key.

// src/client.ts
import { createOpenAI } from '@ai-sdk/openai';
import 'dotenv/config';

export const n4n = createOpenAI({
  baseURL: 'https://api.n4n.ai/v1',
  apiKey: process.env.N4N_API_KEY,
});

export const sonnet = n4n('anthropic/claude-3.5-sonnet');

The model string anthropic/claude-3.5-sonnet is the n4n.ai routing identifier. You can swap it for openai/gpt-4o or any of the 240+ models without changing your tool code.

Define tools with Zod schemas

The Vercel AI SDK expects tools shaped as tool({ description, parameters, execute }). Keep schemas tight — Claude 3.5 Sonnet respects them well, but loose schemas invite hallucinated arguments.

// src/tools.ts
import { tool } from 'ai';
import { z } from 'zod';

export const getWeather = tool({
  parameters: z.object({
    latitude: z.number().min(-90).max(90),
    longitude: z.number().min(-180).max(180),
  }),
  execute: async ({ latitude, longitude }) => {
    // In production, call a real weather API. Here we simulate.
    const tempC = 15 + Math.random() * 15; // 15-30°C
    const conditions = ['clear', 'cloudy', 'rainy', 'windy'][Math.floor(Math.random() * 4)];
    return { temperatureC: Math.round(tempC * 10) / 10, conditions };
  },
});

export const convertTemperature = tool({
  parameters: z.object({
    value: z.number(),
    fromUnit: z.enum(['celsius', 'fahrenheit']),
    toUnit: z.enum(['celsius', 'fahrenheit']),
  }),
  execute: async ({ value, fromUnit, toUnit }) => {
    if (fromUnit === toUnit) return { value, unit: toUnit };
    const converted = fromUnit === 'celsius'
      ? value * 9/5 + 32
      : (value - 32) * 5/9;
    return { value: Math.round(converted * 10) / 10, unit: toUnit };
  },
});

export const tools = { getWeather, convertTemperature };

Build the agent loop

streamText with maxSteps handles the tool-calling loop automatically. Set maxSteps to at least the maximum number of sequential tool calls you expect — here, two (weather then conversion).

// src/agent.ts
import { streamText, CoreMessage } from 'ai';
import { sonnet } from './client';
import { tools } from './tools';

export async function runAgent(messages: CoreMessage[]) {
  const result = streamText({
    model: sonnet,
    tools,
    messages,
    maxSteps: 3,
    system: `You are a weather assistant. 
- Always use getWeather for current conditions. 
- Use convertTemperature when the user asks for a different unit.
- Report temperatures to one decimal place.`,
  });

  return result;
}

Wire up a CLI entry point

// src/index.ts
import { runAgent } from './agent';
import { CoreMessage } from 'ai';

async function main() {
  const args = process.argv.slice(2);
  if (args.length === 0) {
    console.log('Usage: npx tsx src/index.ts "Your question here"');
    process.exit(1);
  }

  const userQuery = args.join(' ');
  const messages: CoreMessage[] = [{ role: 'user', content: userQuery }];

  console.log(`\nUser: ${userQuery}\n`);
  console.log('Assistant: ');

  const result = await runAgent(messages);

  for await (const chunk of result.textStream) {
    process.stdout.write(chunk);
  }
  console.log('\n');

  // Optional: inspect tool calls for debugging
  const toolCalls = await result.toolCalls;
  if (toolCalls.length > 0) {
    console.log('--- Tool calls ---');
    for (const call of toolCalls) {
      console.log(`${call.toolName}(${JSON.stringify(call.args)}) => ${JSON.stringify(call.result)}`);
    }
  }
}

main().catch(console.error);

Run it

Add a script to package.json:

{
  "scripts": {
    "dev": "tsx src/index.ts"
  }
}

Set your API key and run:

export N4N_API_KEY=your-key-here
npm run dev -- "What's the weather in San Francisco in Fahrenheit?"

Expected output (values will differ):

User: What's the weather in San Francisco in Fahrenheit?

Assistant: I'll check the weather in San Francisco and convert it to Fahrenheit for you.

The current weather in San Francisco is 18.2°C with cloudy conditions. Converting to Fahrenheit: 64.8°F.

--- Tool calls ---
getWeather({"latitude":37.7749,"longitude":-122.4194}) => {"temperatureC":18.2,"conditions":"cloudy"}
convertTemperature({"value":18.2,"fromUnit":"celsius","toUnit":"fahrenheit"}) => {"value":64.8,"unit":"fahrenheit"}

Notice the two-step chain: the model inferred San Francisco’s coordinates, called getWeather, then passed the Celsius result to convertTemperature. That’s the multi-step loop working.

Handle provider failures gracefully

n4n.ai automatically falls back when a provider is rate-limited or degraded. You don’t need extra code for this, but you should handle the case where all providers fail. The SDK surfaces this as a standard error.

// src/agent.ts (updated)
import { streamText, CoreMessage } from 'ai';
import { sonnet } from './client';
import { tools } from './tools';

export async function runAgent(messages: CoreMessage[]) {
  try {
    const result = streamText({
      model: sonnet,
      tools,
      messages,
      maxSteps: 3,
      system: `You are a weather assistant. 
- Always use getWeather for current conditions. 
- Use convertTemperature when the user asks for a different unit.
- Report temperatures to one decimal place.`,
    });
    return result;
  } catch (err) {
    if (err instanceof Error && err.message.includes('rate limit')) {
      throw new Error('All providers rate-limited. Try again in a moment.');
    }
    throw err;
  }
}

The gateway also forwards provider cache-control hints. If you enable caching on your n4n.ai dashboard, repeated identical tool calls (like checking the same coordinates twice) can hit the provider cache. You’ll see x-cache: hit in the response headers if you log them.

Stream tool calls in real time

For a better UX, show tool calls as they happen instead of waiting for the final text. The toolCallStream gives you each invocation and result.

// src/index.ts (streaming version)
import { runAgent } from './agent';
import { CoreMessage } from 'ai';

async function main() {
  const args = process.argv.slice(2);
  if (args.length === 0) {
    console.log('Usage: npx tsx src/index.ts "Your question here"');
    process.exit(1);
  }

  const userQuery = args.join(' ');
  const messages: CoreMessage[] = [{ role: 'user', content: userQuery }];

  console.log(`\nUser: ${userQuery}\n`);

  const result = await runAgent(messages);

  // Stream tool calls
  for await (const call of result.toolCallStream) {
    console.log(`🔧 ${call.toolName}(${JSON.stringify(call.args)})`);
    const toolResult = await call.result;
    console.log(`   → ${JSON.stringify(toolResult)}`);
  }

  // Stream final text
  console.log('\nAssistant: ');
  for await (const chunk of result.textStream) {
    process.stdout.write(chunk);
  }
  console.log('\n');
}

main().catch(console.error);

Run the same query again:

User: What's the weather in San Francisco in Fahrenheit?

🔧 getWeather({"latitude":37.7749,"longitude":-122.4194})
   → {"temperatureC":18.2,"conditions":"cloudy"}
🔧 convertTemperature({"value":18.2,"fromUnit":"celsius","toUnit":"fahrenheit"})
   → {"value":64.8,"unit":"fahrenheit"}

Assistant: The current weather in San Francisco is 18.2°C (64.8°F) with cloudy conditions.

Add a second tool: geocoding

Hardcoding coordinates is brittle. Add a geocoding tool so the model can resolve place names itself.

// src/tools.ts (add to existing exports)
export const geocode = tool({
  parameters: z.object({
    place: z.string().min(1),
  }),
  execute: async ({ place }) => {
    // Simulated. Replace with Nominatim, Google Maps, etc.
    const known: Record<string, { lat: number; lon: number }> = {
      'san francisco': { lat: 37.7749, lon: -122.4194 },
      'new york': { lat: 40.7128, lon: -74.0060 },
      'london': { lat: 51.5074, lon: -0.1278 },
      'tokyo': { lat: 35.6762, lon: 139.6503 },
    };
    const key = place.toLowerCase();
    if (known[key]) return known[key];
    // Fallback: approximate center of the string hash
    const hash = key.split('').reduce((a, c) => a + c.charCodeAt(0), 0);
    return { lat: (hash % 180) - 90, lon: (hash % 360) - 180 };
  },
});

export const tools = { getWeather, convertTemperature, geocode };

Update the system prompt to mention the new tool:

// src/agent.ts (updated system prompt)
system: `You are a weather assistant. 
- Use geocode to resolve place names to coordinates.
- Always use getWeather for current conditions. 
- Use convertTemperature when the user asks for a different unit.
- Report temperatures to one decimal place.`,

Now the same query works without you knowing coordinates:

User: What's the weather in Tokyo in Fahrenheit?

🔧 geocode({"place":"Tokyo"})
   → {"lat":35.6762,"lon":139.6503}
🔧 getWeather({"latitude":35.6762,"longitude":139.6503})
   → {"temperatureC":22.5,"conditions":"clear"}
🔧 convertTemperature({"value":22.5,"fromUnit":"celsius","toUnit":"fahrenheit"})
   → {"value":72.5,"unit":"fahrenheit"}

Assistant: The current weather in Tokyo is 22.5°C (72.5°F) with clear conditions.

Three steps, all automatic. The model chose the right sequence: geocode → weather → convert.

Token usage and cost tracking

n4n.ai meters per-token usage across providers. Access it via the usage promise on the result.

// src/index.ts (add after textStream)
const usage = await result.usage;
console.log('\n--- Usage ---');
console.log(`Prompt tokens: ${usage.promptTokens}`);
console.log(`Completion tokens: ${usage.completionTokens}`);
console.log(`Total tokens: ${usage.totalTokens}`);

Output:

--- Usage ---
Prompt tokens: 1,247
Completion tokens: 312
Total tokens: 1,559

This is the gateway-level count. It matches what you’ll see on your n4n.ai dashboard and lets you build cost dashboards per user, per feature, or per conversation.

Common pitfalls

Forgetting maxSteps — The default is 1. Without it, the model calls one tool and stops, leaving the conversion undone.

Loose Zod schemas — If latitude accepts z.number() without bounds, the model may invent latitude: 999. Constrain inputs.

Not handling toolCallStream errors — A tool can throw. Wrap execute in try/catch and return a structured error the model can reason about:

execute: async ({ latitude, longitude }) => {
  try {
    return await fetchWeather(latitude, longitude);
  } catch (e) {
    return { error: 'Weather service unavailable', code: 'WEATHER_DOWN' };
  }
}

Assuming provider-specific tool formats — The Vercel AI SDK normalizes tool calling across OpenAI, Anthropic, and others. n4n.ai’s OpenAI-compatible endpoint means you write tools once. Don’t hand-craft Anthropic’s tool_use blocks.

What’s next

• Replace the simulated weather and geocode tools with real APIs (OpenWeatherMap, Nominatim).
• Add a searchWeb tool for questions beyond weather.
• Persist conversations with result.messages to build multi-turn chat.
• Route different model tiers per task: anthropic/claude-3.5-haiku for simple lookups, Sonnet for complex reasoning.

The pattern stays the same: define tools with Zod, point the SDK at https://api.n4n.ai/v1, and let streamText orchestrate the loop. You get provider diversity, automatic fallback, and unified metering without changing your application code.