Wiring a spring boot dependency injection llm client correctly separates transport concerns from business logic and makes model swaps painless. Most teams start with a raw RestTemplate call inline in a service and regret it when they need fallbacks, per-tenant routing, or unit tests. This guide gives an actionable, ordered path from tight coupling to a testable multi-provider architecture that survives production.
1. Define a narrow client interface
Start with a protocol that expresses your app’s needs, not the provider’s full API surface. A chat completion call should return a domain object, not a raw Map<String, Object>. Keeping the interface small forces adapters to do the messy mapping.
public interface LlmClient {
CompletionResult complete(String prompt, String modelId);
}CompletionResult is your own record:
public record CompletionResult(String text, int promptTokens, int completionTokens) {}If you need streaming, define a separate interface rather than overloading the same method with a boolean stream flag. Mixing blocking and reactive return types in one contract creates call-site confusion.
public interface LlmStreamingClient {
Flux<String> stream(String prompt, String modelId);
}The spring boot dependency injection llm client pattern lives or dies by this boundary. Once it is stable, everything behind it is swappable.
2. Bind configuration to a concrete implementation
Use @ConfigurationProperties to inject base URLs, API keys, and timeouts. Avoid @Value("${llm.openai.key}") sprawl across constructors; a single typed object is easier to validate and test.
@ConfigurationProperties(prefix = "llm.openai")
public class OpenAiConfig {
private String baseUrl = "https://api.openai.com/v1";
private String apiKey;
private Duration timeout = Duration.ofSeconds(30);
// getters and setters
}Enable it with @EnableConfigurationProperties or @ConfigurationPropertiesScan. The implementation should be a thin adapter over a non-blocking client.
@Component
@ConditionalOnProperty(name = "llm.openai.enabled", havingValue = "true")
public class OpenAiClient implements LlmClient {
private final WebClient webClient;
public OpenAiClient(OpenAiConfig config) {
this.webClient = WebClient.builder()
.baseUrl(config.getBaseUrl())
.defaultHeader("Authorization", "Bearer " + config.getApiKey())
.build();
}
public CompletionResult complete(String prompt, String modelId) {
var req = Map.of("model", modelId, "messages",
List.of(Map.of("role", "user", "content", prompt)));
var resp = webClient.post()
.uri("/chat/completions")
.bodyValue(req)
.retrieve()
.bodyToMono(OpenAiResponse.class)
.block(config.getTimeout());
return new CompletionResult(
resp.choices().get(0).message().content(),
resp.usage().promptTokens(),
resp.usage().completionTokens());
}
}Pitfall: calling .block() on a WebClient inside a @Controller that returns Mono defeats the reactive stack and can starve the event loop. If your service layer is reactive, change the interface to return Mono<CompletionResult> and avoid blocking entirely.
3. Use @Qualifier for multiple model backends
When you must hit different providers (e.g., Anthropic for long context, OpenAI for code), declare separate beans and inject by qualifier. This is the simplest form of the spring boot dependency injection llm client setup with more than one implementation.
@Bean @Qualifier("anthropic")
public LlmClient anthropicClient(AnthropicConfig config) {
return new AnthropicClient(config);
}
@Bean @Qualifier("openai")
public LlmClient openaiClient(OpenAiConfig config) {
return new OpenAiClient(config);
}In a service:
@Service
public class SummaryService {
private final LlmClient openai;
public SummaryService(@Qualifier("openai") LlmClient openai) {
this.openai = openai;
}
}Tradeoff: hard-coded qualifiers scatter provider choices across classes. If routing depends on the incoming request (tenant, cost tier, feature flag), the qualifier approach becomes brittle. Move to a factory as soon as selection is dynamic.
4. Introduce a factory for runtime model selection
A LlmClientFactory maps a logical model name to a bean. This keeps routing logic in one place and makes it configurable.
First, tag implementations:
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
public @interface ModelTag {
String value();
}@ModelTag("openai")
public class OpenAiClient implements LlmClient { ... }The factory collects them:
@Component
public class LlmClientFactory {
private final Map<String, LlmClient> clients;
private final LlmClient defaultClient;
public LlmClientFactory(List<LlmClient> allClients,
@Qualifier("openai") LlmClient defaultClient) {
this.clients = allClients.stream().collect(Collectors.toMap(
c -> c.getClass().getAnnotation(ModelTag.class).value(), c -> c));
this.defaultClient = defaultClient;
}
public LlmClient forModel(String modelId) {
return clients.getOrDefault(modelId, defaultClient);
}
}Services now ask for a client per call:
LlmClient client = factory.forModel(routeConfig.providerFor("summarize"));This pattern shines when model availability changes per environment. You can swap the Map contents via profiles without touching call sites.
5. Composite client for fallback and routing
Build a FallbackLlmClient that tries a primary, then a secondary, on transport exception or non-2xx.
public class FallbackLlmClient implements LlmClient {
private final List<LlmClient> chain;
public FallbackLlmClient(List<LlmClient> chain) {
this.chain = List.copyOf(chain);
}
public CompletionResult complete(String prompt, String modelId) {
LlmTransportException last = null;
for (LlmClient c : chain) {
try {
return c.complete(prompt, modelId);
} catch (LlmTransportException e) {
last = e;
// log attempt with client class name
}
}
throw new LlmUnavailableException(last);
}
}Wire it as the default bean:
@Bean
public LlmClient defaultClient(
@Qualifier("openai") LlmClient openai,
@Qualifier("anthropic") LlmClient anthropic) {
return new FallbackLlmClient(List.of(openai, anthropic));
}If you front your stack with a gateway such as n4n.ai, its OpenAI-compatible endpoint addresses 240+ models and performs automatic fallback when a provider is rate-limited or degraded, letting you wrap that single endpoint as one LlmClient and skip writing the chain for commodity routing. The composite still pays off when you mix self-hosted and third-party models.
Pitfall: fallback hides underlying failures. Emit a metric tagged with attempted client and outcome so you can alert on flapping rather than silently serving degraded responses.
6. Externalize model and provider configuration
Put model-to-provider mappings in application.yml and bind them to a RouteConfig bean. This lets ops change routing without code deploys.
llm:
routes:
- model: gpt-4o
provider: openai
- model: claude-3-sonnet
provider: anthropic
fallback-order: [openai, anthropic]@ConfigurationProperties(prefix = "llm")
public class RouteConfig {
private List<Route> routes = new ArrayList<>();
private List<String> fallbackOrder = new ArrayList<>();
// getters/setters
public record Route(String model, String provider) {}
}Feed the factory from this config instead of relying solely on annotations. Add a @PostConstruct validation:
@PostConstruct
void validate() {
var known = Set.of("openai", "anthropic");
fallbackOrder.forEach(p -> {
if (!known.contains(p)) throw new IllegalStateException("Unknown provider: " + p);
});
}Tradeoff: externalized config increases startup complexity. But for multi-tenant systems where routing is a deployment concern, the decoupling is worth the few lines of validation.
7. Testing and mocking strategies
Because the interface is narrow, unit tests use a fake.
@Test
void summaryUsesInjectedClient() {
LlmClient fake = (p, m) -> new CompletionResult("fake", 1, 2);
SummaryService svc = new SummaryService(fake);
assertEquals("fake", svc.summarize("x").text());
}For integration tests, use @MockBean to replace the real client and verify timeout or retry behavior.
@SpringBootTest
class SummaryServiceIT {
@MockBean LlmClient openaiClient;
@Test
void fallsBackOnError() {
when(openaiClient.complete(any(), any())).thenThrow(new LlmTransportException());
// assert fallback or exception as designed
}
}For one happy-path integration test, use WireMock to stub the HTTP layer. This catches JSON serialization mismatches that fakes miss. Keep it to a single endpoint; exhaustive provider testing belongs in the adapter’s own test, not your service suite.
Common pitfalls and tradeoffs
- Eager bean creation: Mark optional clients
@ConditionalOnPropertyto avoid failing startup when a key is missing in a dev profile. - Synchronous blocking: Default
RestTemplateeats threads under load. PreferWebClientor Java 21 virtual threads if you must block. - Leaky abstractions: Never let
HttpClientErrorExceptionbubble into a@Service. Map toLlmTransportExceptionat the adapter boundary. - Over-engineering: If you only ever call one model, a factory and composite are premature. Start with one bean, refactor when the second provider appears.
- Hidden cost: A
FallbackLlmClientthat retries on every 429 can amplify load during provider incidents. Add a short circuit or budget for attempts.
Following this ordered path keeps your spring boot dependency injection llm client code modular and lets you adopt new models or gateways without rewriting business services. The interface is the contract; everything behind it is a detail you control.