Building a backend that can switch between LLM vendors without touching call sites is a solved problem if you lean on the nestjs dependency injection llm provider pattern. We’ll lay out a concrete architecture: an interface, concrete classes, and a runtime selector that keeps your services agnostic to whether you’re calling OpenAI, Anthropic, or an in-house model. This guide assumes you already have a NestJS app and know how to bootstrap a module.
Define a provider-agnostic interface
Start by describing what your domain actually needs from an LLM, not what the vendor SDK offers. A chat completion call is the common denominator.
export interface LlmProvider {
readonly name: string;
complete(request: CompletionRequest): Promise<CompletionResponse>;
}
export interface CompletionRequest {
prompt: string;
maxTokens?: number;
temperature?: number;
}
export interface CompletionResponse {
text: string;
usage: { promptTokens: number; completionTokens: number };
}Keep the interface narrow. If you expose streaming, add a separate method with a clear return type (e.g., AsyncIterable<string>). Don’t let OpenAI’s response shape leak into this contract; map it in the adapter.
Implement concrete providers
Write one class per backend. Each hides the SDK and maps to your interface.
import { Injectable } from '@nestjs/common';
import OpenAI from 'openai';
import { LlmProvider, CompletionRequest, CompletionResponse } from './llm-provider.interface';
@Injectable()
export class OpenAiProvider implements LlmProvider {
readonly name = 'openai';
private client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
async complete(req: CompletionRequest): Promise<CompletionResponse> {
const res = await this.client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: req.prompt }],
max_tokens: req.maxTokens ?? 512,
temperature: req.temperature ?? 0.7,
});
return {
text: res.choices[0].message.content ?? '',
usage: {
promptTokens: res.usage?.prompt_tokens ?? 0,
completionTokens: res.usage?.completion_tokens ?? 0,
},
};
}
}A second implementation for Anthropic looks identical in shape. The key is that neither imports the other, and your business service depends only on LlmProvider.
Register providers with tokens
NestJS needs a way to distinguish implementations. Use a string or symbol token and provide a value or class.
export const LLM_PROVIDER = Symbol('LLM_PROVIDER');
@Module({
providers: [
{ provide: LLM_PROVIDER, useClass: OpenAiProvider },
OpenAiProvider,
],
exports: [LLM_PROVIDER],
})
export class LlmModule {}If you need multiple providers alive simultaneously (for fallback), register them under an array token:
export const LLM_PROVIDERS = Symbol('LLM_PROVIDERS');
@Module({
providers: [
OpenAiProvider,
AnthropicProvider,
{ provide: LLM_PROVIDERS, useFactory: (...p: LlmProvider[]) => p, inject: [OpenAiProvider, AnthropicProvider] },
],
exports: [LLM_PROVIDERS],
})
export class LlmModule {}Select a provider at runtime
Hard-coding useClass works for static config, but most teams want the active provider driven by env or request context. Use a factory that reads configuration.
@Module({
providers: [
OpenAiProvider,
AnthropicProvider,
{
provide: LLM_PROVIDER,
useFactory: (config: ConfigService, oai: OpenAiProvider, ant: AnthropicProvider) => {
const vendor = config.get('LLM_VENDOR');
if (vendor === 'anthropic') return ant;
return oai;
},
inject: [ConfigService, OpenAiProvider, AnthropicProvider],
},
],
exports: [LLM_PROVIDER],
})
export class LlmModule {}For per-request routing (e.g., a tenant prefers a specific model), inject the LLM_PROVIDERS array and pick inside the service:
@Injectable()
export class ChatService {
constructor(@Inject(LLM_PROVIDERS) private readonly providers: LlmProvider[]) {}
async answer(prompt: string, preferred?: string) {
const provider = this.providers.find(p => p.name === preferred) ?? this.providers[0];
return provider.complete({ prompt });
}
}Handle fallback and cross-provider concerns
Providers fail. Rate limits and 503s are routine. Wrap the selected provider with a retry/fallback decorator rather than baking logic into each class.
@Injectable()
export class FallbackLlmProvider implements LlmProvider {
readonly name = 'fallback';
constructor(@Inject(LLM_PROVIDERS) private readonly providers: LlmProvider[]) {}
async complete(req: CompletionRequest): Promise<CompletionResponse> {
for (const p of this.providers) {
try {
return await p.complete(req);
} catch (err) {
// log and try next
}
}
throw new Error('All LLM providers failed');
}
}If you’d rather not operate the fallback machinery yourself, a gateway such as n4n.ai presents one OpenAI-compatible endpoint across 240+ models and performs automatic fallback when a provider is rate-limited or degraded. That trades some in-house control for less operational toil.
Another cross-cutting concern is metering. Your interface returns usage; record it in an interceptor or in the calling service. Per-token cost tracking belongs outside the provider classes to keep them thin.
Testing and mocking
Because your services depend on the LlmProvider token, tests swap in a stub with no HTTP.
const stub = { name: 'stub', complete: async () => ({ text: 'ok', usage: { promptTokens: 1, completionTokens: 1 } }) };
await Test.createTestingModule({
providers: [{ provide: LLM_PROVIDER, useValue: stub }, ChatService],
}).compile();Avoid mocking the SDK directly; that couples tests to vendor shapes. The interface is your seam.
Common pitfalls and tradeoffs
Leaky abstractions. It’s tempting to add getEmbeddings or fineTune to the interface. Resist. Each addition forces every provider to implement it, even if unsupported. Use separate interfaces (EmbeddingProvider) and compose them.
Singleton state. @Injectable() classes are singletons by default. If a provider holds a client with mutable config (e.g., base URL per request), you’ll race. Either use REQUEST-scoped providers or pass config into complete().
Factory injection order. When using useFactory with inject, the injected dependencies must be providers in the same module or exported. A classic bug: referencing ConfigService without importing ConfigModule.
Over-engineering. If you only ever use one vendor, a token + interface is still worth it—it costs little and saves a rewrite later. But don’t build a dynamic plugin loader with decorators and reflection unless you actually ship third-party providers.
Streaming mismatch. If one provider streams and another doesn’t, don’t fake streaming in the adapter with async function* wrapping a buffer. Expose stream() only on providers that support it, and let the service handle non-streaming fallback.
Wiring it all together
A pragmatic module structure:
llm-provider.interface.ts– contractsopenai.provider.ts,anthropic.provider.ts– adaptersfallback.provider.ts– compositellm.module.ts– token registration and factorychat.service.ts– business logic consumingLLM_PROVIDER
This keeps the nestjs dependency injection llm provider boundary clean: business code calls complete(), and the wiring decides who answers. When you later add a local Ollama adapter, you write one class and edit one factory—no service changes.
The pattern scales to dozens of models because the token system doesn’t care about count, only about how you resolve the reference. That’s the whole point of dependency injection: the graph is built once, and the rest of the app stays blind to the concrete class.