Building a Spring Boot service that talks to an LLM provider is straightforward until you need to test it. The spring boot mockwebserver llm testing pattern gives you a deterministic way to simulate chat completions, rate limits, and streaming without burning tokens or flaking on network hiccups. This guide walks through a complete setup using OkHttp’s MockWebServer and Spring’s RestClient, end to end.
Step 1: Scaffold the project and dependencies
Start with a standard Spring Boot 3.2+ application. You need the web starter (for RestClient) and MockWebServer on the test classpath. I prefer RestClient over WebClient for LLM calls: it’s synchronous by default, maps JSON cleanly, and avoids reactor complexity unless you specifically need streaming.
Add these to your build.gradle:
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
testImplementation 'com.squareup.okhttp3:mockwebserver:4.12.0'
}If you use Maven, the equivalent is okhttp3:mockwebserver in <scope>test</scope>. Do not add the OpenAI Java SDK—wrapping the HTTP calls yourself keeps the surface area small and makes the spring boot mockwebserver llm testing approach trivial because you control the serialization.
Step 2: Define the LLM client configuration
Externalize the base URL. In application.yml put:
llm:
base-url: https://api.openai.com/v1
api-key: ${LLM_API_KEY}Create a @Configuration that builds a RestClient scoped to that base URL:
@Configuration
public class LlmConfig {
@Bean
public RestClient llmRestClient(@Value("${llm.base-url}") String baseUrl,
@Value("${llm.api-key}") String apiKey) {
return RestClient.builder()
.baseUrl(baseUrl)
.defaultHeader("Authorization", "Bearer " + apiKey)
.build();
}
}This bean is what we will redirect to MockWebServer later. Keeping it as a plain RestClient bean (rather than a @Service that internally builds its own) means tests can reuse the exact production wiring.
Step 3: Point the client at MockWebServer in tests
Spin up MockWebServer as a JUnit static member. Use @DynamicPropertySource to override llm.base-url so the RestClient targets the mock instead of a real provider. If your production route goes through a gateway such as n4n.ai—which exposes one OpenAI-compatible endpoint covering 240+ models with automatic fallback—you would normally set llm.base-url=https://api.n4n.ai/v1; the test simply swaps that for the local mock.
@SpringBootTest
class LlmServiceTest {
static MockWebServer mockServer = new MockWebServer();
@DynamicPropertySource
static void props(DynamicPropertyRegistry r) {
r.add("llm.base-url", () -> mockServer.url("/").toString());
r.add("llm.api-key", () -> "test-key");
}
@AfterAll
static void shutdown() throws IOException {
mockServer.shutdown();
}
}Now every request from the RestClient hits the mock. This is the core of spring boot mockwebserver llm testing: no live network, no token spend, fully reproducible.
Step 4: Write the service that calls chat completions
Define minimal record types for the OpenAI-compatible request/response shape you care about:
public record ChatMessage(String role, String content) {}
public record ChatRequest(String model, List<ChatMessage> messages) {}
public record ChatChoice(Message message) {
public record Message(String role, String content) {}
}
public record ChatResponse(List<ChatChoice> choices) {}The service method:
@Service
public class LlmService {
private final RestClient client;
public LlmService(RestClient llmRestClient) {
this.client = llmRestClient;
}
public String complete(String model, String prompt) {
var req = new ChatRequest(model, List.of(new ChatMessage("user", prompt)));
var resp = client.post()
.uri("/chat/completions")
.body(req)
.retrieve()
.body(ChatResponse.class);
return resp.choices().get(0).message().content();
}
}Note the relative URI /chat/completions. Because the base URL is overridden in tests, this resolves to the mock.
Step 5: Simulate a successful completion
In your test, enqueue a canned JSON response and assert the service parses it:
@Test
void returnsContentFromMock() {
mockServer.enqueue(new MockResponse()
.setHeader("Content-Type", "application/json")
.setBody("""
{
"choices": [
{ "message": { "role": "assistant", "content": "42" } }
]
}
"""));
var result = service.complete("gpt-4o-mini", "What is 6*7?");
assertThat(result).isEqualTo("42");
RecordedRequest recorded = mockServer.takeRequest();
assertThat(recorded.getPath()).isEqualTo("/chat/completions");
assertThat(recorded.getHeader("Authorization")).startsWith("Bearer ");
}This validates both serialization of the outgoing request and deserialization of the incoming one. Running ./gradlew test should pass with zero external calls.
Step 6: Test error and rate-limit handling
Real LLM gateways return 429 or 500. Your code should handle them. Enqueue a 429 with a Retry-After header and verify your retry logic (here assuming a simple manual loop or @Retryable):
@Test
void retriesOn429() {
mockServer.enqueue(new MockResponse().setResponseCode(429)
.setHeader("Retry-After", "0"));
mockServer.enqueue(new MockResponse()
.setBody("""
{ "choices": [ { "message": { "role":"assistant", "content":"ok" } } ] }
"""));
String out = service.complete("model", "hi");
assertThat(out).isEqualTo("ok");
assertThat(mockServer.getRequestCount()).isEqualTo(2);
}If you use Resilience4j, configure the test with a low retry count and assert the fallback or exception. The spring boot mockwebserver llm testing setup makes these scenarios deterministic—no waiting on real backoff timers.
Step 7: Test streaming responses
Many LLM integrations use Server-Sent Events. MockWebServer can emit chunked text:
@Test
void streamsTokens() {
mockServer.enqueue(new MockResponse()
.setHeader("Content-Type", "text/event-stream")
.setBody("""
data: {"choices":[{"delta":{"content":"Hello"}}]}\n\n
data: {"choices":[{"delta":{"content":" world"}}]}\n\n
data: [DONE]\n\n
"""));
// assuming service.stream(prompt) returns Flux<String> or similar
List<String> chunks = service.stream("model", "hi").collectList().block();
assertThat(chunks).containsExactly("Hello", " world");
}Implement the streaming client with RestClient and a custom BodyExtractor for SSE, or use WebClient if you already adopted reactor. The mock doesn’t care; it just ships bytes.
Step 8: Verify success and integrate into CI
Run the suite:
./gradlew test --tests "*LlmServiceTest"All tests should be green with no network egress. To prove isolation, disconnect Wi-Fi and re-run; they still pass. Wire this into your CI pipeline as a required gate. Because the spring boot mockwebserver llm testing pattern uses only local sockets, it runs fast and is parallel-safe if you give each test class its own MockWebServer instance.
Beyond unit coverage, keep one nightly @IntegrationTest that hits a real provider (or a gateway with per-token metering) to catch contract drift. But the day-to-day PR checks should never leave your machine.
Step 9: Capture request bodies for contract assertions
A subtle bug is sending the wrong model string or malformed messages. Use mockServer.takeRequest().getBody().readUtf8() and assert against expected JSON:
String body = mockServer.takeRequest().getBody().readUtf8();
assertThat(body).contains("\"model\":\"gpt-4o-mini\"");
assertThat(body).contains("\"role\":\"user\"");This catches regressions in your mapping layer without mocking the whole RestClient. Combined with schema snapshots, you get a lightweight contract test against the OpenAI-compatible shape.
Wrapping up the test pyramid
MockWebServer is not a substitute for a real integration test, but it should be your default for logic, retries, and parsing. The spring boot mockwebserver llm testing workflow keeps builds hermetic and lets you simulate impossible-to-reproduce provider states (degraded caches, partial streams) in milliseconds. Once this scaffold is in place, adding coverage for a new model or a new gateway header is a two-line enqueue.