Most teams bolt on a second LLM provider by copying their OpenAI wrapper and renaming fields. A well-structured typescript generics llm client eliminates that duplication: you define request and response shapes once and let the compiler enforce provider differences at the edges. This guide walks through a concrete pattern for building such a client without sacrificing type safety or ergonomics.
1. Define a provider-agnostic request core
Start by extracting the fields every chat API agrees on. OpenAI, Anthropic, and open-weight servers all accept a list of messages, a model identifier, and sampling controls.
interface BaseChatRequest {
model: string;
messages: { role: 'system' | 'user' | 'assistant'; content: string }[];
temperature?: number;
max_tokens?: number;
}Wrap this in a generic that accepts provider-specific extensions. This is the first lever of a typescript generics llm client: the base stays stable, the extra params are parameterized.
type ChatRequest<Extra = Record<string, never>> = BaseChatRequest & Extra;2. Encode provider differences with mapped extras
Each provider diverges in optional fields. OpenAI exposes top_p and stop; Anthropic uses top_k and pulls system prompts to a top-level system string. Model these as a discriminated map.
interface OpenAIExtra {
top_p?: number;
stop?: string[];
response_format?: { type: 'json_object' };
}
interface AnthropicExtra {
top_k?: number;
system?: string;
metadata?: { user_id?: string };
}
type ProviderExtras = {
openai: OpenAIExtra;
anthropic: AnthropicExtra;
};
type Provider = keyof ProviderExtras;Now ChatRequest<ProviderExtras['anthropic']> is a precise type. The compiler rejects an OpenAI call that passes top_k, and vice versa. This catches mismatches at build time instead of in production logs.
Avoid classical inheritance here. A BaseProvider class with subclasses leads to overloaded method signatures and instanceof checks at runtime. The generic map keeps differences in the type system and leaves runtime as a plain switch.
Pitfall: optional vs required
Anthropic mandates max_tokens; OpenAI does not. Use Required<Pick<BaseChatRequest, 'max_tokens'>> in the Anthropic extra intersection if you want strictness:
type AnthropicRequest = ChatRequest<
AnthropicExtra & Required<Pick<BaseChatRequest, 'max_tokens'>>
>;3. Normalize response shapes with generics
Raw responses differ: OpenAI returns choices[0].message.content, Anthropic returns content[0].text. Define a normalized view and keep the raw payload for callers who need it.
interface NormalizedResponse {
id: string;
text: string;
usage: { prompt_tokens: number; completion_tokens: number };
}
type ChatResponse<P extends Provider> = {
raw: RawResponse<P>;
normalized: NormalizedResponse;
};
// RawResponse is a mapped type over provider SDKs
type RawResponse<P extends Provider> = P extends 'openai'
? OpenAIResponse
: AnthropicResponse;A typescript generics llm client uses this dual shape to give you safe access to normalized.text while preserving raw for debugging or provider-specific post-processing.
Write small mappers at the boundary:
function normalizeOpenAI(res: OpenAIResponse): NormalizedResponse {
return {
id: res.id,
text: res.choices[0]?.message?.content ?? '',
usage: res.usage
};
}4. Streaming without erasing types
Streaming endpoints return chunks. OpenAI sends choices[0].delta.content; Anthropic sends delta.text. Define a generic chunk and an async iterable method.
type ChatChunk<P extends Provider> = {
raw: RawChunk<P>;
delta: string;
};
async function* streamChat<P extends Provider>(
provider: P,
req: ChatRequest<ProviderExtras[P]>
): AsyncIterable<ChatChunk<P>> {
// implementation delegates to provider SDK, yields mapped chunks
}Callers iterate without any:
for await (const chunk of streamChat('openai', { model: 'gpt-4o', messages })) {
process.stdout.write(chunk.delta);
}The generic ensures req cannot contain Anthropic-only fields when provider is 'openai'.
5. Build the unified client interface
Expose a single object that hides transport details. Generics propagate from method signatures.
interface LLMClient {
complete: <P extends Provider>(
provider: P,
req: ChatRequest<ProviderExtras[P]>
) => Promise<ChatResponse<P>>;
stream: <P extends Provider>(
provider: P,
req: ChatRequest<ProviderExtras[P]>
) => AsyncIterable<ChatChunk<P>>;
}
const client: LLMClient = {
async complete(provider, req) {
// switch on provider, call SDK, return normalized
},
async *stream(provider, req) {
// similar
}
};Usage shows the typescript generics llm client in action:
const oai = await client.complete('openai', {
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'hi' }],
top_p: 0.9
});
const ant = await client.complete('anthropic', {
model: 'claude-3-5-sonnet',
messages: [{ role: 'user', content: 'hi' }],
system: 'Be terse.'
});This design lets you swap SDK versions or add a new provider by extending ProviderExtras and RawResponse. The rest of your codebase stays untouched.
Tradeoff: type complexity vs coverage
Heavy generics produce cryptic errors when a caller omits a required field. Keep the mapped types small and document each provider block. If you support more than four providers, consider codegen from OpenAPI schemas rather than hand-written unions.
6. Routing through a multi-provider gateway
When you don’t want to maintain per-provider fallback logic, a gateway consolidates the surface area. A service like n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models and automatically falls back when a provider is rate-limited. Your typescript generics llm client can model the gateway as a single provider whose extra params accept routing hints and cache-control forwards.
interface GatewayExtra {
'x-routing'?: { prefer: string[] };
cache_control?: { type: 'ephemeral' };
}
type GatewayRequest = ChatRequest<GatewayExtra>;
// usage:
await client.complete('gateway', {
model: 'anthropic/claude-3.5-sonnet',
messages,
cache_control: { type: 'ephemeral' }
});The gateway honors client routing directives and forwards provider cache-control hints, so the generic layer stays thin. You lose fine-grained provider types but gain resilience.
7. Common pitfalls and tradeoffs
Leaky abstractions. If you expose raw everywhere, callers couple to provider quirks and the generic buys little. Restrict raw access to a logging boundary.
Over-constrained generics. Writing <P extends Provider> on every method is correct, but nesting conditional types inside return values can explode inference. Use named mapped types (RawResponse<P>) instead of inline P extends 'openai' ? ... : ... in three places.
SDK drift. Provider types change monthly. Put all provider SDK imports behind a /internal/providers module so a breaking change touches one file.
Type performance. Large unions with intersecting extras slow down tsc on big codebases. If you hit this, precompute response types with interface rather than type conditional chains.
Streaming cancellation. AsyncIterable doesn’t natively handle AbortSignal. Extend your client method to accept { signal: AbortSignal } and pass it to the SDK call.
Error typing. Provider error shapes differ (error.message vs error.error.message). Define a NormalizedError and map in the same boundary layer as responses.
Actionable path
- Write
BaseChatRequestandChatRequest<Extra>. - Add
ProviderExtrasmap for each vendor you use. - Define
RawResponse/RawChunkand normalized shapes. - Implement
LLMClientwithcompleteandstreamgenerics. - If using a gateway, add a
gatewayprovider entry with routing extras. - Isolate provider SDK calls; keep generics at the public boundary.
- Add mapper functions and unit tests that assert normalized output per provider.
A disciplined typescript generics llm client turns multi-provider chaos into a compile-time checked surface. You write one call site, the compiler proves the provider match, and runtime stays boring.