Streaming tool call results with Vercel AI SDK

Streaming tool call results with Vercel AI SDK transforms how users perceive latency. Instead of waiting for a model to finish reasoning, execute tools, and generate a final response, you can surface each tool invocation and its output as it happens. This guide walks through building a complete streaming pipeline with useChat, streamText, and server-side tool handlers that emit partial results incrementally.

Step 1: Set up the project and dependencies

Create a Next.js App Router project with the AI SDK and a provider of your choice. The examples use OpenAI, but the patterns apply to any provider compatible with the AI SDK.

npx create-next-app@latest streaming-tools --typescript --tailwind --eslint --app --src-dir --import-alias "@/*"
cd streaming-tools
npm install ai @ai-sdk/openai zod

The ai package contains streamText, useChat, and tool utilities. zod validates tool parameters and results.

Step 2: Define tools with streaming-friendly schemas

Tools that benefit from streaming typically involve multi-stage operations: file searches, API pagination, database scans, or long-running computations. Define each tool with a description, parameters schema, and an execute function that yields partial results.

// src/lib/tools.ts
import { tool } from 'ai';
import { z } from 'zod';

export const searchDocumentation = tool({
  parameters: z.object({
    query: z.string().describe('Search query'),
    maxResults: z.number().default(5).describe('Maximum results to return'),
  }),
  execute: async function* ({ query, maxResults }) {
    // Simulate a multi-stage search with progressive results
    const stages = [
      { stage: 'index', message: 'Loading search index...' },
      { stage: 'query', message: `Executing query: "${query}"` },
      { stage: 'filter', message: 'Filtering results by relevance...' },
      { stage: 'complete', message: 'Search complete' },
    ];

    for (const stage of stages) {
      yield { type: 'progress', stage: stage.stage, message: stage.message };
      await new Promise(r => setTimeout(r, 300)); // Simulate work
    }

    // Final results
    const results = [
      { title: 'Getting Started', url: '/docs/getting-started', score: 0.95 },
      { title: 'Authentication', url: '/docs/auth', score: 0.87 },
      { title: 'API Reference', url: '/docs/api', score: 0.82 },
    ].slice(0, maxResults);

    yield { type: 'result', results };
  },
});

export const fetchUserMetrics = tool({
  parameters: z.object({
    userId: z.string().describe('User identifier'),
    metrics: z.array(z.string()).describe('Metric names to fetch'),
  }),
  execute: async function* ({ userId, metrics }) {
    yield { type: 'progress', stage: 'connect', message: 'Connecting to metrics store...' };
    await new Promise(r => setTimeout(r, 200));

    const data: Record<string, number> = {};
    for (const metric of metrics) {
      yield { type: 'progress', stage: 'fetch', message: `Fetching ${metric}...`, metric };
      await new Promise(r => setTimeout(r, 150));
      data[metric] = Math.floor(Math.random() * 1000);
    }

    yield { type: 'result', data };
  },
});

Key points: the execute function is an async generator (async function*). Each yield becomes a discrete chunk the client can render immediately. The type field discriminates between progress updates and final results.

Step 3: Create the streaming API route

The route handler uses streamText with the tools defined above. Crucially, pass toolCallStreaming: true to enable streaming of tool invocations and their generator outputs.

// src/app/api/chat/route.ts
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { searchDocumentation, fetchUserMetrics } from '@/lib/tools';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o'),
    messages,
    tools: {
      searchDocumentation,
      fetchUserMetrics,
    },
    toolCallStreaming: true,
    // Optional: constrain tool choice or provide system context
    system: 'You are a helpful assistant with access to documentation search and user metrics. Stream tool progress to the user.',
  });

  return result.toDataStreamResponse({
    // Send tool call and tool result chunks as they arrive
    sendReasoning: true,
    sendSources: true,
  });
}

toDataStreamResponse serializes the stream into the AI SDK’s wire format. The sendReasoning and sendSources options include model reasoning traces and source attributions when available.

Step 4: Build the client-side chat interface

Use useChat from @ai-sdk/react to consume the streaming response. The hook exposes messages, input, handleSubmit, and critically, data — an array of arbitrary objects emitted by tool generators.

// src/app/components/Chat.tsx
'use client';

import { useChat } from '@ai-sdk/react';
import { useState } from 'react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit, data, status, error } = useChat({
    api: '/api/chat',
    onError: e => console.error('Chat error:', e),
  });

  const [expandedToolCalls, setExpandedToolCalls] = useState<Set<string>>(new Set());

  const toggleToolCall = (toolCallId: string) => {
    const next = new Set(expandedToolCalls);
    if (next.has(toolCallId)) next.delete(toolCallId);
    else next.add(toolCallId);
    setExpandedToolCalls(next);
  };

  return (
    <div className="flex flex-col h-[calc(100vh-4rem)] p-4 gap-4">
      <div className="flex-1 overflow-y-auto space-y-4">
        {messages.map(message => (
          <Message key={message.id} message={message} />
        ))}
        {data && data.length > 0 && (
          <ToolCallStream data={data} expanded={expandedToolCalls} onToggle={toggleToolCall} />
        )}
        {status === 'streaming' && <TypingIndicator />}
        {error && <ErrorMessage error={error} />}
      </div>
      <form onSubmit={handleSubmit} className="flex gap-2">
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask about docs or user metrics..."
          className="flex-1 px-4 py-2 border rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
          disabled={status === 'submitting'}
        />
        <button
          type="submit"
          disabled={status === 'submitting' || !input.trim()}
          className="px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700 disabled:opacity-50"
        >
          Send
        </button>
      </form>
    </div>
  );
}

function Message({ message }: { message: { id: string; role: string; content: string; toolInvocations?: any[] } }) {
  return (
    <div className={`flex gap-3 ${message.role === 'user' ? 'justify-end' : ''}`}>
      <div
        className={`max-w-[70%] p-3 rounded-lg ${
          message.role === 'user' ? 'bg-blue-600 text-white' : 'bg-gray-100'
        }`}
      >
        <p className="whitespace-pre-wrap">{message.content}</p>
        {message.toolInvocations?.map(invocation => (
          <ToolInvocationBadge key={invocation.toolCallId} invocation={invocation} />
        ))}
      </div>
    </div>
  );
}

function ToolInvocationBadge({ invocation }: { invocation: { toolCallId: string; toolName: string; state: string; args: any; result?: any } }) {
  const isComplete = invocation.state === 'result';
  return (
    <details className="mt-2 text-xs" open={isComplete}>
      <summary className="font-mono text-gray-500 cursor-pointer">
        🔧 {invocation.toolName} ({invocation.state})
      </summary>
      <pre className="mt-1 p-2 bg-gray-800 text-green-300 rounded overflow-auto">
        {JSON.stringify(isComplete ? invocation.result : invocation.args, null, 2)}
      </pre>
    </details>
  );
}

function ToolCallStream({ data, expanded, onToggle }: { data: any[]; expanded: Set<string>; onToggle: (id: string) => void }) {
  // Group streaming chunks by toolCallId
  const toolStreams = new Map<string, any[]>();
  data.forEach(chunk => {
    if (chunk.toolCallId) {
      const arr = toolStreams.get(chunk.toolCallId) || [];
      arr.push(chunk);
      toolStreams.set(chunk.toolCallId, arr);
    }
  });

  return (
    <div className="border-t pt-4">
      <h3 className="text-sm font-medium text-gray-500 mb-2">Tool execution stream</h3>
      <div className="space-y-2">
        {Array.from(toolStreams.entries()).map(([toolCallId, chunks]) => (
          <ToolCallStreamCard
            key={toolCallId}
            toolCallId={toolCallId}
            chunks={chunks}
            isExpanded={expanded.has(toolCallId)}
            onToggle={() => onToggle(toolCallId)}
          />
        ))}
      </div>
    </div>
  );
}

function ToolCallStreamCard({ toolCallId, chunks, isExpanded, onToggle }: { toolCallId: string; chunks: any[]; isExpanded: boolean; onToggle: () => void }) {
  const latest = chunks[chunks.length - 1];
  const toolName = latest?.toolName || 'unknown';

  return (
    <div className="border rounded-lg p-3 bg-gray-50">
      <button onClick={onToggle} className="flex items-center gap-2 w-full text-left">
        <span className="font-mono text-sm">{toolName}</span>
        <span className="text-xs text-gray-500">{chunks.length} updates</span>
        <span className="ml-auto">{isExpanded ? '▼' : '▶'}</span>
      </button>
      {isExpanded && (
        <div className="mt-2 space-y-1 font-mono text-xs">
          {chunks.map((chunk, i) => (
            <div key={i} className={`p-1 rounded ${chunk.type === 'progress' ? 'bg-yellow-50' : 'bg-green-50'}`}>
              <span className="text-gray-400">[{chunk.stage || chunk.type}]</span>{' '}
              {chunk.message || JSON.stringify(chunk.result || chunk.data || chunk)}
              {chunk.metric && <span className="text-blue-600 ml-2">({chunk.metric})</span>}
            </div>
          ))}
        </div>
      )}
    </div>
  );
}

function TypingIndicator() {
  return <div className="text-sm text-gray-500 animate-pulse">Assistant is thinking…</div>;
}

function ErrorMessage({ error }: { error: Error }) {
  return <div className="text-sm text-red-600">Error: {error.message}</div>;
}

The data array from useChat contains every chunk yielded by tool generators. The ToolCallStream component groups chunks by toolCallId and renders a collapsible timeline. This gives users visibility into each stage of a tool’s execution without blocking the final response.

Step 5: Handle tool result types on the client

The AI SDK emits specific chunk types for tool calls. Understanding the shape lets you build richer UIs. The wire format includes:

• tool-call — model initiated a tool invocation
• tool-call-streaming-start — generator started yielding
• tool-call-delta — partial result from generator
• tool-call-result — generator completed
• tool-result — final tool result attached to assistant message

You can inspect raw chunks by adding a onChunk callback to useChat:

const { data } = useChat({
  api: '/api/chat',
  onChunk: chunk => {
    if (chunk.type === 'tool-call-delta') {
      console.log('Tool delta:', chunk.toolCallId, chunk.delta);
    }
  },
});

For most applications, the data array (which aggregates tool-call-delta chunks) is sufficient. Each element carries toolCallId, toolName, type (progress | result), and payload fields.

Step 6: Verify end-to-end streaming

Start the dev server and test the flow:

npm run dev

Open http://localhost:3000. Try these prompts:

1. “Search the documentation for authentication” — triggers searchDocumentation with progress stages.
2. “Get metrics for user_123: requests, errors, latency” — triggers fetchUserMetrics with per-metric yields.
3. “Find docs on API rate limits and also get metrics for user_456: throughput” — exercises parallel tool calls.

You should see:

• The assistant message appears incrementally.
• A “Tool execution stream” section appears below messages while tools run.
• Each tool card expands to show progress updates in real time.
• Final results render in the assistant message once tools complete.

Check the Network tab: the /api/chat response is a text/plain stream with newline-delimited JSON chunks. Each chunk corresponds to a model token, tool call, or tool delta.

Step 7: Add error handling and timeouts

Production code needs guardrails. Wrap tool generators in try/catch and yield error chunks. Enforce per-tool timeouts to prevent hung streams.

// src/lib/tools.ts (updated execute wrapper)
import { tool } from 'ai';
import { z } from 'zod';

function withTimeout<T>(promise: Promise<T>, ms: number, label: string): Promise<T> {
  return Promise.race([
    promise,
    new Promise<never>((_, reject) =>
      setTimeout(() => reject(new Error(`${label} timed out after ${ms}ms`)), ms)
    ),
  ]);
}

export const resilientSearch = tool({
  parameters: z.object({ query: z.string() }),
  execute: async function* ({ query }) {
    try {
      yield { type: 'progress', stage: 'start', message: 'Starting search...' };
      const results = await withTimeout(performSearch(query), 10000, 'search');
      yield { type: 'result', results };
    } catch (err) {
      yield { type: 'error', message: err instanceof Error ? err.message : 'Search failed' };
    }
  },
});

async function performSearch(query: string) {
  // Real implementation here
  await new Promise(r => setTimeout(r, 500));
  return [{ title: 'Result 1', url: '/1' }, { title: 'Result 2', url: '/2' }];
}

On the client, render type === 'error' chunks with distinct styling so users know a tool failed without breaking the conversation.

Step 8: Optimize for perceived latency

Streaming tool results is as much a UX problem as an engineering one. Three patterns improve perceived performance:

Optimistic tool invocation — Show a “tool started” badge immediately when the model emits a tool call, before the first generator yield. The tool-call chunk arrives before tool-call-streaming-start.

Progressive disclosure — Collapse completed tool streams by default. Users expand only what they care about. The ToolCallStreamCard component above implements this.

Parallel tool execution — The AI SDK runs multiple tool calls concurrently when the model invokes them in the same turn. Your generator functions should be stateless and side-effect-free to support this safely.

Step 9: Deploy and monitor

Deploy to Vercel or any Node.js platform. The streaming response works behind Vercel’s Edge Functions with maxDuration configured. For longer tools, consider offloading to a queue and polling, or using WebSockets for bidirectional streams.

Monitor these signals:

• Time to first tool delta (should be < 500ms)
• Tool completion rate (success vs timeout vs error)
• Stream duration percentiles (p50, p95, p99)

If you route traffic through an inference gateway like n4n.ai, you gain automatic fallback when a provider degrades and per-token metering across all tool-augmented calls — useful for cost attribution when tools invoke external APIs.

Step 10: Extend with custom chunk types

The generator yield type is any. Define a discriminated union for your application to get TypeScript safety across server and client.

// src/lib/stream-types.ts
export type ToolProgressChunk = {
  type: 'progress';
  stage: string;
  message: string;
  metadata?: Record<string, unknown>;
};

export type ToolResultChunk<T> = {
  type: 'result';
  data: T;
};

export type ToolErrorChunk = {
  type: 'error';
  message: string;
  code?: string;
};

export type ToolChunk<T> = ToolProgressChunk | ToolResultChunk<T> | ToolErrorChunk;

Use ToolChunk<YourResultType> as the generator return type. On the client, the data array elements will be ToolChunk<unknown> — narrow with type guards before rendering.

This pattern scales from simple lookup tools to complex multi-agent workflows. The key insight: treat tool execution as a stream, not a blocking RPC. Users get continuous feedback, models stay in the loop longer, and your application feels faster without reducing actual latency.