Parallel tool calls in the Vercel AI SDK explained

Parallel tool calls let a model invoke multiple functions in a single turn instead of waiting for each call to return before deciding the next one. The Vercel AI SDK surfaces this through the maxSteps option and the toolChoice parameter, allowing the model to batch independent operations like fetching weather for three cities or querying a database and calling an external API simultaneously. This cuts round-trip latency roughly in proportion to the number of independent calls you can batch.

How parallel tool calls work in the SDK

When you call streamText or generateText with maxSteps greater than 1, the SDK enters a loop: it sends the conversation to the model, receives tool calls, executes them, feeds results back, and repeats until the model produces a final answer or hits the step limit. If the model returns multiple tool_calls in one response, the SDK executes them concurrently using Promise.all under the hood.

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

const result = streamText({
  model: openai('gpt-4o'),
  tools: {
    getWeather: weatherTool,
    getStockPrice: stockTool,
    searchDocs: searchTool,
  },
  maxSteps: 5,
  prompt: 'Get the weather in NYC, SF, and London, then check AAPL and MSFT prices',
});

The model sees all three tools and decides to call getWeather three times and getStockPrice twice in one response. The SDK fans those five calls out in parallel, waits for all to resolve, then sends the combined results back to the model for the next step.

The toolChoice parameter

You can influence this behavior with toolChoice:

• "auto" (default): model decides whether and which tools to call
• "required": model must call at least one tool
• { type: "function", function: { name: "getWeather" } }: force a specific tool
• "none": disable tools entirely

For parallel calls, "auto" is usually what you want. The model will batch calls when the user’s request clearly decomposes into independent sub-tasks.

Why parallel calls matter for latency

Sequential tool calling adds latency linearly. If each tool takes 800 ms and you need five calls, that’s 4 seconds of pure waiting before the model can synthesize an answer. Parallel execution bounds the wall-clock time to the slowest single call plus overhead — roughly 800–1,000 ms for the same five calls.

This matters most for:

• Dashboard-style queries: “Show me revenue, user count, and error rate for the last 7 days” — three independent DB queries
• Multi-source retrieval: Vector search + keyword search + SQL lookup for a RAG pipeline
• External API fan-out: Checking inventory across three warehouse systems, or pricing across multiple vendors

The SDK’s maxSteps defaults to 1, which forces sequential behavior. Bumping it to 3–5 unlocks parallelism for most real workloads without risking infinite loops.

Concrete example: building a travel planner

A travel planner needs flights, hotels, and weather for a date range. These are independent — perfect for parallel calls.

// tools/travel.ts
import { tool } from 'ai';
import { z } from 'zod';

export const searchFlights = tool({
  parameters: z.object({
    origin: z.string(),
    destination: z.string(),
    departDate: z.string(),
    returnDate: z.string().optional(),
  }),
  execute: async ({ origin, destination, departDate, returnDate }) => {
    // Call your flight API here
    return flightApi.search({ origin, destination, departDate, returnDate });
  },
});

export const searchHotels = tool({
  parameters: z.object({
    city: z.string(),
    checkIn: z.string(),
    checkOut: z.string(),
    guests: z.number().default(1),
  }),
  execute: async ({ city, checkIn, checkOut, guests }) => {
    return hotelApi.search({ city, checkIn, checkOut, guests });
  },
});

export const getWeatherForecast = tool({
  parameters: z.object({
    city: z.string(),
    startDate: z.string(),
    endDate: z.string(),
  }),
  execute: async ({ city, startDate, endDate }) => {
    return weatherApi.forecast({ city, startDate, endDate });
  },
});

// app/api/travel/route.ts
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { searchFlights, searchHotels, getWeatherForecast } from '@/tools/travel';

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    tools: { searchFlights, searchHotels, getWeatherForecast },
    maxSteps: 3,
    system: `You are a travel planner. When users ask for trip options, 
      call searchFlights, searchHotels, and getWeatherForecast in parallel 
      for each destination they mention.`,
    messages,
  });

  return result.toDataStreamResponse();
}

User asks: “Plan a trip to Tokyo and Kyoto for March 15–22, flying from SFO.”

Model response (single turn):

{
  "tool_calls": [
    { "name": "searchFlights", "arguments": { "origin": "SFO", "destination": "NRT", "departDate": "2025-03-15", "returnDate": "2025-03-22" }},
    { "name": "searchFlights", "arguments": { "origin": "SFO", "destination": "KIX", "departDate": "2025-03-15", "returnDate": "2025-03-22" }},
    { "name": "searchHotels", "arguments": { "city": "Tokyo", "checkIn": "2025-03-15", "checkOut": "2025-03-22", "guests": 1 }},
    { "name": "searchHotels", "arguments": { "city": "Kyoto", "checkIn": "2025-03-15", "checkOut": "2025-03-22", "guests": 1 }},
    { "name": "getWeatherForecast", "arguments": { "city": "Tokyo", "startDate": "2025-03-15", "endDate": "2025-03-22" }},
    { "name": "getWeatherForecast", "arguments": { "city": "Kyoto", "startDate": "2025-03-15", "endDate": "2025-03-22" }}
  ]
}

All six calls fire simultaneously. The SDK collects results, feeds them back, and the model produces the final itinerary in step 2.

Handling partial failures

Parallel execution means some calls succeed while others fail. The SDK passes each tool’s result (or error) back to the model individually. Your tools should return structured error objects rather than throwing, so the model can reason about partial data.

export const searchFlights = tool({
  // ...
  execute: async (args) => {
    try {
      return await flightApi.search(args);
    } catch (err) {
      return { error: true, code: 'FLIGHT_API_ERROR', message: err.message };
    }
  },
});

The model sees { "error": true, "code": "FLIGHT_API_ERROR", ... } and can tell the user “I got hotel and weather data but the flight search failed — want me to retry or proceed with what we have?”

Controlling concurrency

The SDK uses Promise.all by default, which fires everything at once. If you have rate limits or want to bound concurrency, wrap your tool’s execute with a semaphore or use a library like p-limit.

import pLimit from 'p-limit';

const limit = pLimit(3); // max 3 concurrent external calls

export const searchFlights = tool({
  // ...
  execute: (args) => limit(() => flightApi.search(args)),
});

This keeps the model’s parallel intent while protecting downstream services.

Common misconceptions

“MaxSteps enables parallel calls”

maxSteps enables multi-step conversations. Parallel calls happen within a single step when the model returns multiple tool_calls. You need both: maxSteps > 1 for the loop, and a prompt that encourages the model to batch independent calls.

“The SDK automatically parallelizes sequential-looking code”

If you write:

const weather = await getWeather('NYC');
const stocks = await getStocks(['AAPL', 'MSFT']);

That’s your code running sequentially. The SDK only parallelizes when the model emits multiple tool calls in one response. Design your tools and prompts so the model sees independent operations.

“All models support parallel tool calls”

OpenAI models (GPT-4o, GPT-4-turbo) and Anthropic models (Claude 3.5 Sonnet, Opus) support it. Some smaller or older models only emit one tool call per turn. Check the provider’s documentation. The SDK doesn’t polyfill this — if the model returns one call, you get one call.

“Parallel calls always improve latency”

Only if the calls are truly independent and I/O-bound. If tools share a rate-limited connection pool, or if one tool’s output is required for the next, parallelism adds overhead without benefit. Profile your specific tool chain.

“You need special streaming handling”

streamText handles parallel tool calls transparently. The data stream emits tool-call chunks for each call, then tool-result chunks as they resolve (in completion order, not submission order), then continues. Your frontend just renders what arrives.

// Frontend consumption - no special handling needed
const { messages } = useChat({
  onToolCall: ({ toolCall }) => console.log('Started:', toolCall.toolName),
  onToolResult: ({ toolResult }) => console.log('Finished:', toolResult.toolName),
});

Debugging parallel execution

Enable SDK logging to see the fan-out/fan-in:

import { createLogger } from 'ai';

const result = streamText({
  // ...
  experimental_telemetry: {
    isEnabled: true,
    functionId: 'travel-planner',
  },
  onFinish: ({ usage, steps }) => {
    steps.forEach((step, i) => {
      console.log(`Step ${i}: ${step.toolCalls?.length ?? 0} tool calls`);
      step.toolCalls?.forEach(tc => console.log(`  - ${tc.toolName}`));
    });
  },
});

Output:

Step 0: 6 tool calls
  - searchFlights
  - searchFlights
  - searchHotels
  - searchHotels
  - getWeatherForecast
  - getWeatherForecast
Step 1: 0 tool calls

This confirms the model batched all six calls in step 0 and produced the final answer in step 1.

When to avoid parallel calls

• Dependent operations: Tool B needs Tool A’s output. Force sequence with prompt engineering or split into separate steps.
• Rate-limited APIs: If your downstream allows 5 req/s and the model fans out 20 calls, you’ll get 429s. Add concurrency limits or batch in the tool itself.
• Expensive operations: If each call costs significant money or compute, let the user confirm first or use a cheaper model to plan the calls.
• Debugging complexity: Parallel failures are harder to trace. Start sequential, parallelize after you have observability.

Summary

Parallel tool calls in the Vercel AI SDK reduce latency by letting the model invoke multiple independent functions in a single turn. Set maxSteps > 1, design tools that are genuinely independent, and the SDK handles the concurrent execution via Promise.all. The model decides when to batch — your job is giving it the right tools and the right prompt. Profile the actual latency gains in your workload; they’re real but depend entirely on your tool I/O patterns.