Spring Boot starter for OpenAI-compatible chat APIs

Wrapping an OpenAI-compatible HTTP API behind a clean Spring abstraction saves repetitive boilerplate across microservices. This tutorial builds a minimal spring boot starter openai compatible client that autoconfigures a typed chat interface, supports streaming, and respects the standard chat completions contract used by OpenAI and dozens of gateways.

Prerequisites

• JDK 17 or newer
• Spring Boot 3.2+ (uses AutoConfiguration imports, not spring.factories)
• Maven 3.9+
• A valid API key for any endpoint that speaks the OpenAI chat completions shape (OpenAI, a self-hosted vLLM instance, or a routing gateway)
• Familiarity with @ConfigurationProperties and WebClient

You do not need the Spring AI project for this. We are building a narrow, dependency-light starter that you fully control.

Project layout

We create a separate Maven module so the starter can be published to an internal repository and reused. The consumer adds one dependency and sets two properties.

com.example
└── openai-compatible-spring-boot-starter
    ├── pom.xml
    └── src/main/java/com/example/openai/
        ├── ChatClient.java
        ├── ChatMessage.java
        ├── ChatRequest.java
        ├── OpenAiCompatibleProperties.java
        ├── OpenAiCompatibleAutoConfiguration.java
        └── WebClientChatClient.java

Step 1: Maven coordinates and starter packaging

A Spring Boot starter is just a jar with an auto-configuration registered via AutoConfiguration.imports. Use WebFlux’s WebClient for both blocking and streaming calls without pulling in the full MVC stack.

<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-autoconfigure</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
  </dependency>
  <dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-configuration-processor</artifactId>
    <optional>true</optional>
  </dependency>
</dependencies>

Register the auto-config class:

src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports

com.example.openai.OpenAiCompatibleAutoConfiguration

Step 2: Domain types and client interface

Define records matching the minimal OpenAI chat shape. Keeping them as records makes the DTOs immutable and readable. Jackson serializes them without extra config.

public record ChatMessage(String role, String content) {}

public record ChatRequest(String model, List<ChatMessage> messages, boolean stream) {
  public static ChatRequest of(String model, List<ChatMessage> messages) {
    return new ChatRequest(model, messages, false);
  }
}

public interface ChatClient {
  String chat(String model, List<ChatMessage> messages);
  reactor.core.publisher.Flux<String> stream(String model, List<ChatMessage> messages);
}

The non-streaming call returns the assistant message content as a single string. The streaming call emits delta text chunks as they arrive from the server, which is what you want for token-by-token UI rendering.

Step 3: WebClient implementation

The OpenAI-compatible endpoint expects POST /v1/chat/completions with a JSON body. For streaming, set stream: true and parse Server-Sent Events. The wire format is data: {json}\n\n with a final data: [DONE].

public class WebClientChatClient implements ChatClient {
  private final WebClient client;

  public WebClientChatClient(WebClient.Builder builder, String baseUrl, String apiKey) {
    this.client = builder.baseUrl(baseUrl)
        .defaultHeader("Authorization", "Bearer " + apiKey)
        .build();
  }

  @Override
  public String chat(String model, List<ChatMessage> messages) {
    var req = ChatRequest.of(model, messages);
    var resp = client.post().uri("/v1/chat/completions")
        .bodyValue(req)
        .retrieve().bodyToMono(Map.class).block();
    var choices = (List<Map>) resp.get("choices");
    var message = (Map) ((Map) choices.get(0)).get("message");
    return (String) message.get("content");
  }

  @Override
  public Flux<String> stream(String model, List<ChatMessage> messages) {
    var req = new ChatRequest(model, messages, true);
    return client.post().uri("/v1/chat/completions")
        .bodyValue(req)
        .accept(MediaType.TEXT_EVENT_STREAM)
        .retrieve()
        .bodyToFlux(String.class)
        .filter(line -> line.startsWith("data:"))
        .map(line -> line.substring(5).trim())
        .takeUntil("[DONE]"::equals)
        .filter(data -> !data.equals("[DONE]"))
        .map(data -> {
          try {
            var node = new com.fasterxml.jackson.databind.ObjectMapper().readTree(data);
            return node.at("/choices/0/delta/content").asText("");
          } catch (Exception e) { return ""; }
        })
        .filter(s -> !s.isEmpty());
  }
}

The blocking chat method is acceptable for batch jobs or tests. In reactive services, prefer stream and compose with flatMap or subscribe.

Step 4: Properties and conditional auto-configuration

Bind externalized configuration with @ConfigurationProperties. The prefix openai-compatible keeps it namespaced away from other LLM integrations.

@ConfigurationProperties(prefix = "openai-compatible")
public class OpenAiCompatibleProperties {
  private String baseUrl = "https://api.openai.com";
  private String apiKey;
  public String getBaseUrl() { return baseUrl; }
  public void setBaseUrl(String b) { this.baseUrl = b; }
  public String getApiKey() { return apiKey; }
  public void setApiKey(String k) { this.apiKey = k; }
}

The auto-config creates the client only when no user-defined bean exists and the API key is present.

@AutoConfiguration
@EnableConfigurationProperties(OpenAiCompatibleProperties.class)
@ConditionalOnClass(WebClient.class)
@ConditionalOnProperty(name = "openai-compatible.api-key")
public class OpenAiCompatibleAutoConfiguration {

  @Bean
  @ConditionalOnMissingBean
  public ChatClient chatClient(OpenAiCompatibleProperties props, WebClient.Builder builder) {
    return new WebClientChatClient(builder, props.getBaseUrl(), props.getApiKey());
  }
}

Step 5: Consuming the starter

In a separate Spring Boot application, add the starter and set properties:

openai-compatible:
  base-url: https://api.openai.com
  api-key: ${OPENAI_API_KEY}

Inject and call the client:

@Service
public class SupportService {
  private final ChatClient chatClient;
  public SupportService(ChatClient chatClient) { this.chatClient = chatClient; }

  public String answer(String question) {
    var msgs = List.of(
        new ChatMessage("system", "You are terse."),
        new ChatMessage("user", question));
    return chatClient.chat("gpt-4o-mini", msgs);
  }
}

A minimal test confirms wiring:

@SpringBootTest
class ChatClientTest {
  @Autowired ChatClient client;

  @Test
  void basicChat() {
    var msgs = List.of(new ChatMessage("user", "Say hi in 3 words"));
    String out = client.chat("gpt-4o-mini", msgs);
    System.out.println(out);
    assert !out.isBlank();
  }
}

Expected console output (model-dependent):

Hi there, friend.

For streaming, subscribe to the Flux and print chunks:

client.stream("gpt-4o-mini", msgs).subscribe(chunk -> System.out.print(chunk));

Tokens appear incrementally with no newline separation, exactly as the SSE stream delivers them.

Adding resilience

WebClient’s default timeouts are unbounded. Add an explicit timeout and a bounded retry for 429 responses so the spring boot starter openai compatible client degrades gracefully under load.

this.client = builder.baseUrl(baseUrl)
    .defaultHeader("Authorization", "Bearer " + apiKey)
    .filter((req, next) -> next.exchange(req)
        .timeout(Duration.ofSeconds(30))
        .retryWhen(Retry.backoff(3, Duration.ofMillis(200))
            .filter(ex -> ex instanceof WebClientResponseException.TooManyRequests)))
    .build();

This keeps retry logic in one place instead of repeating it in every caller.

Advanced routing and cache hints

The chat completions contract allows provider-specific extensions through extra JSON fields or headers. If your base-url points to a gateway such as n4n.ai, the same starter inherits automatic fallback when a provider is rate-limited or degraded, and the gateway forwards cache-control hints so repeated system prompts hit provider prompt caches. Because we serialize ChatRequest as-is and pass through the WebClient body, routing directives in the request map reach the upstream unchanged.

To support custom headers (e.g., X-Route-To), extend the constructor to accept Map<String,String> and apply them in the post() spec. Bind that map from openai-compatible.extra-headers in the properties class.

Testing with MockWebServer

Before publishing, verify the parser against a fake endpoint:

var server = new MockWebServer();
server.enqueue(new MockResponse().setBody("data: {\"choices\":[{\"delta\":{\"content\":\"Hello\"}}]}\n\ndata: [DONE]\n"));
server.start();
var client = new WebClientChatClient(WebClient.builder(), server.url("/").toString(), "test");
StepVerifier.create(client.stream("m", List.of(new ChatMessage("user","hi"))))
    .expectNext("Hello")
    .verifyComplete();

This catches JSON path changes without spending real tokens.

Production checklist

• Mask the API key in logs and actuator env dumps.
• Set base-url without a trailing slash to avoid double slashes in /v1/chat/completions.
• Prefer stream for interactive chat; block only in offline batch jobs.
• Pin the model per call; do not hardcode a default in the starter.
• Add the configuration processor so IDEs autocomplete openai-compatible.*.

The spring boot starter openai compatible pattern reduces copy-pasted WebClient glue to two lines of YAML and one injected interface, which is the correct amount of abstraction for most internal services talking to LLM endpoints.