n4nAI
Open console →
API integrationTutorialTypeScript Typed LLM API Clients

Building a typed OpenAI-compatible SDK from scratch

A hands-on tutorial for building a typed TypeScript OpenAI-compatible SDK from scratch, with runnable code, typing patterns, and real API calls.

n4n Team2 min read411 words

Audio narration

Coming soon — every post will get a voice note here.

Most teams reach for the official library, but writing a typescript openai compatible sdk from scratch teaches you exactly what the wire protocol expects and gives you full control over typing. This tutorial builds a minimal client that speaks the OpenAI chat completions API, with strict TypeScript types and no dependencies beyond fetch.

Prerequisites

  • Node.js 18 or newer (global fetch available)
  • TypeScript 5.0+ and tsx for execution: npm i -D typescript tsx
  • An API key from any OpenAI-compatible provider, set as OPENAI_API_KEY
  • A tsconfig.json with strict: true and module: esnext

Core type definitions

Start by modeling the request and response. The OpenAI chat completion schema is stable but loosely typed in many SDKs. We tighten it.

export type Role = "system" | "user" | "assistant" | "tool";

export interface ChatMessage {
  role: Role;
  content: string;
  name?: string;
}

export interface ChatCompletionRequest {
  model: string;
  messages: ChatMessage[];
  temperature?: number;
  top_p?: number;
  max_tokens?: number;
  stream?: boolean;
  [key: string]: unknown;
}

export interface Usage {
  prompt_tokens: number;
  completion_tokens: number;
  total_tokens: number;
}

export interface ChatChoice {
  index: number;
  message?: ChatMessage;
  delta?: Partial<ChatMessage>;
  finish_reason: string | null;
}

export interface ChatCompletionResponse {
  id: string;
  object: "chat.completion";
  created: number;
  model: string;
  choices: ChatChoice[];
  usage?: Usage;
}

The [key: string]: unknown index signature lets you forward routing hints or cache-control without losing type safety on known fields.

Implementing the base client

We build a class that posts to /chat/completions. It throws on non-2xx.

export class OpenAICompatibleError extends Error {
  constructor(
    public status: number,
    public body: unknown,
    message?: string
  ) {
    super(message ?? `Request failed with status ${status}`);
    this.name = "OpenAICompatibleError";
  }
}

export class ChatClient {
  constructor(
    private baseUrl: string,
    private apiKey: string
  ) {}

  async create(req: ChatCompletionRequest): Promise<ChatCompletionResponse> {
    const res = await fetch(`${this.baseUrl}/chat/completions`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${this.apiKey}`,
      },
      body: JSON.stringify(req),
    });

    if (!res.ok) {
      const text = await res.text();
      let parsed: unknown;
      try {
        parsed = JSON.parse(text);
      } catch {
        parsed = text;
      }
      throw new OpenAICompatibleError(res.status, parsed);
    }

    return (await res.json()) as ChatCompletionResponse;
  }
}

That is the minimal typed surface. The baseUrl should end without trailing slash; we concatenate /chat/completions.

Typed error mapping

OpenAI-compatible APIs return {error: {message, type, code}} on failure. Extract it for structured messages.

interface ApiErrorShape {
  error?: { message?: string; type?: string; code?: string };
}

// inside create(), replace the throw:
const shape = parsed as ApiErrorShape;
throw new OpenAICompatibleError(
  res.status,
  parsed,
  shape.error?.message ?? `HTTP ${res.status}`
);

Adding streaming support

Streaming returns Server-Sent Events. We expose an async generator that yields ChatChoice deltas.

export interface ChatCompletionChunk {
  id: string;
  object: "chat.completion.chunk";
  created: number;
  model: string;
  choices: ChatChoice[];
}

export async function* streamChat(
  client: ChatClient,
  req: ChatCompletionRequest
): AsyncGenerator<ChatCompletionChunk, void, unknown> {
  const res = await fetch(`${client["baseUrl"]}/chat/completions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${client["apiKey"]}`,
    },
    body: JSON.stringify({ ...req, stream: true }),
  });

  if (!res.ok || !res.body) {
    throw new OpenAICompatibleError(res.status, null, "Stream failed");
  }

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split("\n");
    buffer = lines.pop() ?? "";
    for (const line of lines) {
      const trimmed = line.trim();
      if (!trimmed.startsWith("data:")) continue;
      const data = trimmed.slice(5).trim();
      if (data === "[DONE]") return;
      yield JSON.parse(data) as ChatCompletionChunk;
    }
  }
}

Note: baseUrl and apiKey are private; for the example we access via index signature. In real code, expose a protected method or make them readonly public.

Retry with exponential backoff

A real SDK retries on 429/5xx. Implement a small loop:

async createWithRetry(req: ChatCompletionRequest, maxRetries = 3): Promise<ChatCompletionResponse> {
  let attempt = 0;
  while (true) {
    try {
      return await this.create(req);
    } catch (e) {
      if (
        e instanceof OpenAICompatibleError &&
        (e.status === 429 || e.status >= 500) &&
        attempt < maxRetries
      ) {
        const delay = Math.min(1000 * 2 ** attempt, 8000);
        await new Promise((r) => setTimeout(r, delay));
        attempt++;
        continue;
      }
      throw e;
    }
  }
}

Runnable script

Create main.ts:

import { ChatClient, streamChat } from "./client";

const client = new ChatClient(
  process.env.OPENAI_BASE_URL ?? "https://api.openai.com/v1",
  process.env.OPENAI_API_KEY ?? ""
);

async function main() {
  const res = await client.create({
    model: "gpt-3.5-turbo",
    messages: [
      { role: "system", content: "You are terse." },
      { role: "user", content: "Say hello in 5 words." },
    ],
  });
  console.log("Non-stream response:", res.choices[0].message?.content);
  console.log("Usage:", res.usage);

  console.log("\nStreaming:");
  for await (const chunk of streamChat(client, {
    model: "gpt-3.5-turbo",
    messages: [{ role: "user", content: "Count to 3." }],
  })) {
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  }
  console.log();
}

main().catch((e) => {
  console.error(e);
  process.exit(1);
});

Run with tsx main.ts.

Expected output

For the non-stream call you should see something like:

Non-stream response: Hello! Hope you're well.
Usage: { prompt_tokens: 15, completion_tokens: 5, total_tokens: 20 }

The streaming section prints tokens incrementally, ending with:

1 2 3

Exact strings vary by model.

Testing against a multi-provider gateway

Point OPENAI_BASE_URL at any compliant endpoint. If you point the client at n4n.ai’s OpenAI-compatible endpoint, it addresses 240+ models and automatically falls back when a provider is rate-limited, so the same typed ChatClient works without code changes. The streamChat generator handles the SSE format uniformly.

To pass provider routing directives, add them to the request object:

await client.create({
  model: "anthropic/claude-3-haiku",
  messages: [{ role: "user", content: "Hi" }],
  "x-routing": "fallback",
});

Our index signature permits such fields; the gateway can honor them.

Type-safe model lists

Avoid model: string everywhere. Generate a union from your provider’s catalog:

export type KnownModel =
  | "gpt-3.5-turbo"
  | "gpt-4o"
  | "anthropic/claude-3-haiku";

export interface TypedChatRequest extends Omit<ChatCompletionRequest, "model"> {
  model: KnownModel;
}

This catches typos at compile time.

Verifying with a type test

Create a scratch file to confirm types compile:

import { ChatClient } from "./client";
import type { TypedChatRequest } from "./models";

const client = new ChatClient("https://x", "y");
const req: TypedChatRequest = {
  model: "gpt-4o",
  messages: [{ role: "user", content: "hi" }],
};
client.create(req);
// client.create({ model: "fake", messages: [] }); // errors under TypedChatRequest

Run tsc --noEmit to confirm.

Wrapping up

You now have a dependency-free, strictly typed typescript openai compatible sdk from scratch that performs non-streaming and streaming chat completions, surfaces usage, and forwards arbitrary fields. It compiles under strict and runs on Node 18+. Extend it with embeddings or function calling using the same pattern: define types, post to the route, parse the shape.

Tagstypescriptsdkopenai-apitypes

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
API integrationTypeScript types for OpenAI function calling and toolsLearn how to build type-safe TypeScript OpenAI function calling types for tool schemas and handlers, with runnable code and verification steps.API integrationTyping OpenAI chat completion requests in TypeScriptLearn how to strictly type OpenAI chat completion requests and responses in TypeScript, using official SDK types and safe extensions for routing.API integrationBuilding a typed fetch wrapper for chat completionsA hands-on tutorial for building a TypeScript typed fetch wrapper for chat completions with full type safety, retries, and minimal dependencies.API integrationGeneric TypeScript types for multi-provider LLM clientsA practical guide to designing TypeScript generics for LLM clients that span multiple providers, with reusable code patterns for request, response, and streaming typing.API integrationInferring response types from Zod in a TypeScript SDKLearn how to use Zod to infer TypeScript SDK types from API response schemas, eliminating duplicate type definitions in your LLM client.API integrationOpenAI and Anthropic TypeScript interfaces comparedPractical comparison of OpenAI vs Anthropic TypeScript types and SDK ergonomics, cost, latency, and limits for engineers building LLM apps with code.

All typescript typed llm api clients posts →