Tool-calling breaks in production for boring reasons: a schema mismatch, a missed required parameter, or a parser that chokes on a nested object. Testing tool-calling TypeScript Vitest setups from day one catches these before they reach users, and it’s simpler than most teams assume. This guide walks through a complete test pyramid for function-calling code, from pure unit tests to mocked orchestration and a real-network integration check.
Step 1: Scaffold a TypeScript project with Vitest
Create a fresh directory and initialize a minimal Node project. Vitest runs TypeScript natively, so you don’t need a separate compile step for tests.
mkdir tool-call-tests && cd tool-call-tests
npm init -y
npm install -D typescript vitest @types/node
npx tsc --init --module esnext --moduleResolution bundler --strictAdd a test script to package.json:
{
"scripts": {
"test": "vitest run",
"test:watch": "vitest"
}
}Verify the scaffold by running npx vitest run. You should see “No test files found” and a zero exit code. That’s your baseline.
Step 2: Define the tool and its types
A tool-calling flow has two sides: the JSON schema the model sees, and the TypeScript function that executes the call. Keep them in one module so drift is obvious.
We’ll use Zod for runtime validation—it doubles as a schema source and a guard in the executor.
// src/tools/weather.ts
import { z } from 'zod';
export const weatherSchema = z.object({
lat: z.number().min(-90).max(90),
lon: z.number().min(-180).max(180),
units: z.enum(['metric', 'imperial']).default('metric'),
});
export type WeatherArgs = z.infer<typeof weatherSchema>;
export interface WeatherResult {
temp: number;
condition: string;
}
export async function getWeather(args: WeatherArgs): Promise<WeatherResult> {
// Imagine a fetch to a real API; we stub it for tests.
if (args.lat === 0 && args.lon === 0) {
return { temp: 28, condition: 'clear' };
}
return { temp: 15, condition: 'cloudy' };
}
export const weatherTool = {
name: 'get_weather',
parameters: {
type: 'object',
properties: {
lat: { type: 'number' },
lon: { type: 'number' },
units: { type: 'string', enum: ['metric', 'imperial'] },
},
required: ['lat', 'lon'],
},
};Step 3: Unit test the executor in isolation
The cheapest test is the pure function. Mock the network inside the module if needed, but here the stub is deterministic.
// src/tools/weather.test.ts
import { describe, it, expect } from 'vitest';
import { getWeather, weatherSchema } from './weather';
describe('getWeather executor', () => {
it('returns clear sky at null island', async () => {
const args = weatherSchema.parse({ lat: 0, lon: 0 });
const res = await getWeather(args);
expect(res.condition).toBe('clear');
});
it('defaults to metric units', async () => {
const args = weatherSchema.parse({ lat: 51.5, lon: -0.1 });
expect(args.units).toBe('metric');
await expect(getWeather(args)).resolves.toMatchObject({ temp: 15 });
});
});Run npx vitest run src/tools. Both tests should pass. If they fail, you have a type or logic error before any LLM is involved.
Step 4: Mock the model to test the orchestration loop
Real value comes from testing the loop that sends a user prompt, receives a tool_call, executes it, and sends the result back. Mock the provider SDK so the test is fast and deterministic.
Assume a thin client wrapper around an OpenAI-compatible API:
// src/agent.ts
import { weatherTool, getWeather, WeatherArgs } from './tools/weather';
export async function runAgent(
prompt: string,
chat: (messages: any[]) => Promise<{ tool_calls?: any[]; content?: string }>,
): Promise<string> {
const messages = [{ role: 'user', content: prompt }];
const first = await chat(messages);
if (!first.tool_calls) return first.content ?? '';
for (const call of first.tool_calls) {
if (call.function.name === weatherTool.name) {
const args = JSON.parse(call.function.arguments) as WeatherArgs;
const result = await getWeather(args);
messages.push({ role: 'tool', content: JSON.stringify(result), tool_call_id: call.id });
}
}
const second = await chat(messages);
return second.content ?? '';
}Now test it with vi.fn():
// src/agent.test.ts
import { describe, it, expect, vi } from 'vitest';
import { runAgent } from './agent';
describe('runAgent tool-calling loop', () => {
it('executes weather tool and returns final answer', async () => {
const chat = vi.fn();
chat.mockResolvedValueOnce({
tool_calls: [
{ id: 'call_1', function: { name: 'get_weather', arguments: '{"lat":0,"lon":0}' } },
],
}).mockResolvedValueOnce({ content: 'It is clear at 28 degrees.' });
const out = await runAgent('What is the weather at null island?', chat);
expect(chat).toHaveBeenCalledTimes(2);
expect(chat.mock.calls[1][0]).toContainEqual({
role: 'tool',
content: JSON.stringify({ temp: 28, condition: 'clear' }),
tool_call_id: 'call_1',
});
expect(out).toBe('It is clear at 28 degrees.');
});
it('skips tool when model answers directly', async () => {
const chat = vi.fn().mockResolvedValueOnce({ content: 'I cannot help with that.' });
const out = await runAgent('Tell me a joke', chat);
expect(chat).toHaveBeenCalledTimes(1);
expect(out).toBe('I cannot help with that.');
});
});This validates that your testing tool-calling TypeScript Vitest harness correctly exercises the round-trip without spending a token.
Step 5: Test schema validation and failure paths
Models hallucinate parameters. Your code must reject bad input before calling the upstream API.
// src/tools/weather.test.ts (append)
describe('weatherSchema validation', () => {
it('rejects out-of-range latitude', () => {
expect(() => weatherSchema.parse({ lat: 100, lon: 0 })).toThrow();
});
it('coerces missing units to default', () => {
const parsed = weatherSchema.parse({ lat: 10, lon: 10 });
expect(parsed.units).toBe('metric');
});
it('throws on non-number lon', () => {
expect(() => weatherSchema.parse({ lat: 10, lon: 'east' })).toThrow();
});
});Add a test for the agent when the tool arguments are malformed:
// src/agent.test.ts (append)
it('propagates validation error from tool args', async () => {
const chat = vi.fn().mockResolvedValueOnce({
tool_calls: [{ id: 'c', function: { name: 'get_weather', arguments: '{"lat":999}' } }],
});
// Assuming runAgent wraps getWeather with schema parse; adjust to catch.
await expect(runAgent('bad', chat)).rejects.toThrow();
});If your executor doesn’t validate, add a weatherSchema.parse call inside runAgent before getWeather. That’s the fix.
Step 6: Integration test against a real endpoint
Unit and mock tests prove logic, but not that your request shape matches the provider. Write a skipped integration test that hits a real OpenAI-compatible endpoint when an API key exists.
If you route through a gateway such as n4n.ai, which exposes one OpenAI-compatible endpoint covering 240+ models and honors client routing directives, you can assert that a cache_control hint is forwarded by inspecting response headers in the test.
// src/integration.test.ts
import { describe, it, expect } from 'vitest';
const BASE = process.env.LLM_BASE_URL ?? 'https://api.openai.com/v1';
const KEY = process.env.LLM_API_KEY;
describe.skipIf(!KEY)('live tool-calling integration', () => {
it('returns a tool call for a weather prompt', async () => {
const res = await fetch(`${BASE}/chat/completions`, {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${KEY}`,
'x-cache-control': 'ephemeral', // forwarded by gateways that honor hints
},
body: JSON.stringify({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'Weather at lat 0 lon 0?' }],
tools: [
{
type: 'function',
function: {
name: 'get_weather',
parameters: {
type: 'object',
properties: { lat: { type: 'number' }, lon: { type: 'number' } },
required: ['lat', 'lon'],
},
},
},
],
}),
});
expect(res.status).toBe(200);
const data = await res.json();
expect(data.choices[0].message.tool_calls?.[0]?.function?.name).toBe('get_weather');
});
});Run with LLM_API_KEY=sk-... npx vitest run src/integration.test.ts. Success means the provider understood your schema and emitted a call. Without the key, the suite skips cleanly.
Step 7: Run the full suite and verify success
Consolidate everything:
npx vitest runYou should see output similar to:
✓ src/tools/weather.test.ts (5)
✓ src/agent.test.ts (3)
↓ src/integration.test.ts (skipped)Coverage of the executor and orchestration loop is complete. The testing tool-calling TypeScript Vitest workflow now guards against schema drift, missing parameters, and broken round-trips.
What to add next
- Property-based tests with
fast-checkfor coordinate generation. - Contract tests that snapshot the exact tool schema sent to the model.
- A CI job that runs the integration test against a staging key monthly.
Keep the tests close to the tool definitions; when the schema changes, the test fails loudly. That’s the whole point.