Function calling in Node.js with the OpenAI SDK

Function calling turns an LLM from a text generator into a structured action executor. The openai node.js sdk function calling interface is stable across compatible providers, but the details of schema design and execution loops trip up most first implementations. This guide walks a concrete path from client setup to a resilient production loop.

Install and configure the client

Use the official openai package. It speaks the OpenAI REST contract, so any OpenAI-compatible endpoint works by swapping the baseURL.

npm install openai zod

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  // baseURL: 'https://api.n4n.ai/v1', // optional: OpenAI-compatible gateway
});

Keep the key in env. Don’t hardcode. The same constructor works if you later route through a gateway that adds automatic fallback.

Define tool schemas with strict JSON Schema

The model selects a tool by name and returns arguments as a JSON object. Your schema is the contract. Vague schemas produce malformed args.

const tools = [{
  type: 'function',
  function: {
    name: 'get_weather',
    parameters: {
      type: 'object',
      properties: {
        lat: { type: 'number', description: 'Latitude' },
        lon: { type: 'number', description: 'Longitude' },
        unit: { type: 'string', enum: ['c', 'f'] }
      },
      required: ['lat', 'lon'],
      additionalProperties: false
    }
  }
}];

Validate before execution

The SDK does not validate args against the schema. Use a runtime validator like Zod to catch model mistakes.

import { z } from 'zod';

const WeatherArgs = z.object({
  lat: z.number(),
  lon: z.number(),
  unit: z.enum(['c', 'f']).default('c')
});

If validation fails, return a tool error message to the model so it can self-correct. Don’t throw out of the loop.

Send the first request with tools

A non-streaming call is simplest for debugging. Pass tools and a tool_choice of auto unless you want to force a specific function.

const res = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'What is the weather at 37.77, -122.41?' }],
  tools,
  tool_choice: 'auto'
});

const msg = res.choices[0].message;

If msg.tool_calls is present, the model wants to run functions. Otherwise it answered directly.

Build the execution loop

The core pattern: append the assistant message, execute each tool call, append results as tool role messages, then call again. Stop when no tool calls return.

async function runConversation(userInput: string) {
  const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
    { role: 'user', content: userInput }
  ];

  for (let i = 0; i < 5; i++) {
    const res = await client.chat.completions.create({
      model: 'gpt-4o-mini',
      messages,
      tools,
    });
    const msg = res.choices[0].message;
    messages.push(msg);

    if (!msg.tool_calls) return msg.content;

    for (const call of msg.tool_calls) {
      const args = WeatherArgs.parse(JSON.parse(call.function.arguments));
      const data = await getWeather(args); // your impl
      messages.push({
        role: 'tool',
        tool_call_id: call.id,
        content: JSON.stringify(data)
      });
    }
  }
  throw new Error('Exceeded max tool iterations');
}

Handle parallel tool calls

Models may emit multiple tool_calls in one message. Execute them concurrently, but keep the order of appended tool messages consistent with the call IDs. The API matches by tool_call_id, not array position.

await Promise.all(msg.tool_calls.map(async (call) => {
  const args = WeatherArgs.parse(JSON.parse(call.function.arguments));
  const data = await getWeather(args);
  messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(data) });
}));

Beware: concurrent side effects can cause races if your tools mutate shared state. Serialize when needed.

Streaming changes the loop shape

Streaming keeps latency low but complicates tool call assembly. The stream: true response emits deltas; tool_calls arrive as partial JSON strings across chunks.

const stream = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages,
  tools,
  stream: true
});

let acc = '';
for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta;
  if (delta?.tool_calls?.[0]?.function?.arguments) {
    acc += delta.tool_calls[0].function.arguments;
  }
}

You must buffer and concatenate arguments per tool_call_id. Only after the stream ends do you parse and validate. Tradeoff: you lose the ability to abort a bad tool call mid-stream. For most internal tools, non-streaming is simpler and fast enough.

Choose the right model and tool_choice

Not every model supports tools equally. Smaller models like gpt-4o-mini handle simple schemas; complex nested objects may need gpt-4o. Set tool_choice: { type: 'function', function: { name: '...' } } only when you know a tool must run—otherwise auto lets the model skip tools for pure chat.

Tradeoff: forcing a tool increases latency and cost if the user didn’t need it. The openai node.js sdk function calling loop remains identical across model tiers, so swap freely during testing.

Production concerns: retries and fallback

Networks fail. Providers rate-limit. Wrap the create call with a retry that respects Retry-After and uses exponential backoff. The openai node.js sdk function calling flow is stateless per request, so retries are safe if you haven’t executed side effects yet.

If you route through an OpenAI-compatible gateway such as n4n.ai, you get automatic fallback when a provider is degraded, plus per-token metering and honored cache-control hints—useful when you run the same tool loop across multiple model backends. That said, your execution loop shouldn’t assume a single model; pin model per call based on cost or latency budgets.

Idempotency and side effects

Tool executions must be idempotent or guarded. If a retry happens after a tool ran but before the result posted, you’ll double-execute. Use a dedupe key from tool_call_id plus a request id.

const executed = new Set<string>();
async function execOnce(call: OpenAI.Chat.Completions.ChatCompletionMessageToolCall) {
  if (executed.has(call.id)) return;
  executed.add(call.id);
  // ... run
}

Manage context and token cost

Each tool result is appended to messages and resent on the next call. A 5-step loop with 2KB results multiplies context fast. Trim old tool messages or summarize.

if (messages.length > 20) {
  messages = [messages[0], ...messages.slice(-10)];
}

Also, the openai node.js sdk function calling requests count input tokens for the full schema on every call. Keep tool definitions lean; remove unused properties.

Error handling pattern

Return errors as tool messages, not exceptions. The model can adapt.

try {
  const data = await getWeather(args);
  messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(data) });
} catch (e) {
  messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify({ error: String(e) }) });
}

This keeps the loop alive instead of crashing the whole conversation.

Logging and observability

Log the raw tool_calls and parsed args. When a validation fails, log the original string. This surfaces schema mismatches quickly.

console.log('tool_call', call.function.name, call.function.arguments);

Use correlation IDs per conversation to trace multi-step loops in production dashboards.

Common pitfalls

Schema drift. If you change a tool’s parameters without redeploying the loop, the model will send old shapes. Version your tool names (get_weather_v2).

Ignoring additionalProperties. Set additionalProperties: false to stop the model from injecting unknown fields that break strict validators.

Unbounded loops. Always cap iterations. A confused model can call tools repeatedly.

Returning huge tool outputs. Models have context limits. Summarize or truncate API responses before pushing to messages.

Assuming arguments is valid JSON. It often is, but streaming fragments or partial completions may break JSON.parse. Wrap in try/catch and feed the error back as a tool message.

Forgetting concurrent side effects. Parallel tool calls can double-write to a database. Gate with the idempotency set shown above.

Testing the loop

Write unit tests that mock client.chat.completions.create to return scripted tool_calls. Verify your validator rejects bad args and that the loop terminates.

// pseudo-test
mockCreate.mockResolvedValueOnce({
  choices: [{ message: { tool_calls: [{ id: '1', function: { name: 'get_weather', arguments: '{"lat":1}' } }] } }]
});

Function calling is not magic; it’s a disciplined request-response dance. Get the schemas tight, cap the iterations, validate everything the model sends you, and the openai node.js sdk function calling pattern will scale to real workloads.