n4nAI
Open console →
API integrationHow-toJava LLM API Integration

Unit testing LLM API calls in Java with WireMock

Step-by-step java wiremock llm api testing tutorial: mock LLM endpoints in Java, inject faults, and validate retry and parsing logic safely.

n4n Team3 min read720 words

Audio narration

Coming soon — every post will get a voice note here.

Most Java services call LLMs over HTTP, typically hitting an OpenAI-compatible /v1/chat/completions route. Applying java wiremock llm api testing lets you replace that network dependency with a fast, deterministic stub, so you can test parsing, retries, and error handling without burning tokens or waiting on rate limits.

Step 1: Pull in WireMock and JUnit 5

Add the dependencies to your Maven build. WireMock runs as a standalone JVM process or embedded; embedded is simplest for unit tests because it boots in-process and tears down cleanly.

<dependency>
  <groupId>com.github.tomakehurst</groupId>
  <artifactId>wiremock-jre8</artifactId>
  <version>2.35.0</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.junit.jupiter</groupId>
  <artifactId>junit-jupiter</artifactId>
  <version>5.10.0</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-databind</artifactId>
  <version>2.15.2</version>
</dependency>

If you use Gradle, the equivalent is testImplementation 'com.github.tomakehurst:wiremock-jre8:2.35.0' and the Jackson/ JUnit coordinates above. Keep WireMock in test scope only; it should never ship in your production artifact.

Step 2: Write a minimal LLM client

Keep the client thin. It builds a JSON request, posts it, and maps the response to a record. Below uses java.net.http.HttpClient (standard in Java 11+) and Jackson for binding.

public record ChatMessage(String role, String content) {}
public record ChatRequest(String model, List<ChatMessage> messages) {}
public record ChatResponse(String id, String model, List<Choice> choices) {}
public record Choice(String finish_reason, ChatMessage message) {}

public class LlmClient {
    private final HttpClient http = HttpClient.newHttpClient();
    private final String baseUrl;
    private final String apiKey;
    private final ObjectMapper mapper = new ObjectMapper();

    public LlmClient(String baseUrl, String apiKey) {
        this.baseUrl = baseUrl;
        this.apiKey = apiKey;
    }

    public ChatResponse complete(ChatRequest req) throws Exception {
        var body = mapper.writeValueAsString(req);
        var request = HttpRequest.newBuilder()
            .uri(URI.create(baseUrl + "/v1/chat/completions"))
            .header("Content-Type", "application/json")
            .header("Authorization", "Bearer " + apiKey)
            .POST(HttpRequest.BodyPublishers.ofString(body))
            .build();
        var resp = http.send(request, HttpResponse.BodyHandlers.ofString());
        if (resp.statusCode() != 200) {
            throw new RuntimeException("LLM call failed: " + resp.statusCode());
        }
        return mapper.readValue(resp.body(), ChatResponse.class);
    }
}

This is the code under test. It has three branches worth covering: happy path, non-200 status, and retry/timeout behavior. Note that we do not implement retry here yet; we add it in Step 5 so the earlier tests stay focused.

Step 3: Start WireMock and stub a success response

WireMock listens on a port and returns canned JSON. In a test class, boot it on a random port and configure a stub that matches the POST path. Using dynamicPort() avoids collisions with other tests.

import com.github.tomakehurst.wiremock.WireMockServer;
import static com.github.tomakehurst.wiremock.client.WireMock.*;

class LlmClientTest {
    private WireMockServer wire;
    private LlmClient client;

    @BeforeEach
    void setup() {
        wire = new WireMockServer(options().dynamicPort());
        wire.start();
        var baseUrl = "http://localhost:" + wire.port();
        client = new LlmClient(baseUrl, "test-key");
    }

    @AfterEach
    void teardown() {
        wire.stop();
    }
}

Now stub the happy path. The response shape mirrors OpenAI’s chat completion contract.

void stubSuccess() {
    wire.stubFor(post("/v1/chat/completions")
        .willReturn(aResponse()
            .withHeader("Content-Type", "application/json")
            .withBody("""
                {
                  "id": "chatcmpl-123",
                  "model": "gpt-4o-mini",
                  "choices": [
                    {
                      "finish_reason": "stop",
                      "message": { "role": "assistant", "content": "Hello from stub" }
                    }
                  ]
                }
                """)));
}

Step 4: Write the happy-path test

Call the client and assert the parsed fields. This proves your JSON mapping and URL construction are correct.

@Test
void returnsParsedCompletion() throws Exception {
    stubSuccess();
    var req = new ChatRequest("gpt-4o-mini",
        List.of(new ChatMessage("user", "hi")));
    var resp = client.complete(req);
    assertEquals("chatcmpl-123", resp.id());
    assertEquals("Hello from stub", resp.choices().get(0).message().content());
}

Run with ./mvn test. A green test here means your client serializes the request and deserializes the response correctly against the stubbed contract. If Jackson complains about unknown properties, add @JsonIgnoreProperties(ignoreUnknown = true) to your records.

Step 5: Simulate rate limits and server errors

Production LLM endpoints return 429 or 500 under load. WireMock can return those statuses and headers without any real backend. First, add a retry method to the client:

public ChatResponse completeWithRetry(ChatRequest req, int maxAttempts) throws Exception {
    Exception last = null;
    for (int i = 0; i < maxAttempts; i++) {
        try { return complete(req); }
        catch (RuntimeException e) { last = e; Thread.sleep(100); }
    }
    throw last;
}

Then stub a scenario where the first call is rate-limited and the second succeeds:

@Test
void retriesOn429() throws Exception {
    wire.stubFor(post("/v1/chat/completions")
        .inScenario("retry")
        .whenScenarioStateIs("Started")
        .willReturn(aResponse().withStatus(429).withHeader("Retry-After", "1"))
        .willSetStateTo("Recovered"));
    wire.stubFor(post("/v1/chat/completions")
        .inScenario("retry")
        .whenScenarioStateIs("Recovered")
        .willReturn(aResponse().withBody("""
            {"id":"ok","model":"gpt-4o-mini","choices":[
              {"finish_reason":"stop","message":{"role":"assistant","content":"recovered"}}]}
            """)));
    var resp = client.completeWithRetry(
        new ChatRequest("gpt-4o-mini", List.of()), 2);
    assertEquals("recovered", resp.choices().get(0).message().content());
}

Scenario states in WireMock are the cleanest way to model stateful fault injection. You can also use withFixedDelay(2000) to test client timeouts.

Step 6: Verify fallback routing with a gateway

If your production traffic goes through a gateway such as n4n.ai, which provides automatic fallback when a provider is rate-limited, you can stub a 429 from one route and a 200 from a fallback to confirm your client honors the redirect. WireMock can simulate this by returning a 503 on the primary model, while your client switches model names based on response.

wire.stubFor(post("/v1/chat/completions")
    .withRequestBody(containing("\"model\":\"primary\""))
    .willReturn(aResponse().withStatus(503)));
wire.stubFor(post("/v1/chat/completions")
    .withRequestBody(containing("\"model\":\"fallback\""))
    .willReturn(aResponse().withBody("{\"id\":\"fb\",\"model\":\"fallback\",\"choices\":[]}")));

Your test then asserts that after a 503 the client retries with the fallback model string. This catches routing bugs before they hit real tokens. Gateways that forward provider cache-control hints also let you assert that your Cache-Control header is passed through unmodified.

Step 7: Assert outgoing request shape and headers

A stub can also verify that your client sent the right headers and body. Use verify() and getRequestedFor. This is where java wiremock llm api testing pays off: you lock the contract.

@Test
void sendsAuthHeaderAndModel() throws Exception {
    stubSuccess();
    client.complete(new ChatRequest("gpt-4o-mini", List.of()));
    wire.verify(postRequestedFor(urlEqualTo("/v1/chat/completions"))
        .withHeader("Authorization", equalTo("Bearer test-key"))
        .withRequestBody(containing("\"model\":\"gpt-4o-mini\"")));
}

For stricter checks, use withRequestBody(equalToJson("{\"model\":\"gpt-4o-mini\",\"messages\":[]}")). If you forward cache-control hints (some gateways honor them), check the header is present:

.withHeader("Cache-Control", equalTo("max-age=60"))

Negative testing is just as important: stub a 200 with malformed JSON and assert your client throws a clean exception rather than a Jackson stack trace.

Step 8: Run the suite and confirm deterministic timing

Execute ./mvn test -Dtest=LlmClientTest. All tests should pass in under a second. WireMock runs in-process; no network egress occurs. If you see flaky failures, ensure the server stops between tests (@AfterEach) and ports are dynamic.

A successful run gives you:

  • Proof of correct JSON (de)serialization.
  • Coverage of 429/503 retry and fallback switching.
  • Guaranteed request contract (auth, model, cache headers).

That is the core of java wiremock llm api testing—fast, offline, and exact.

Where to go next

Add scenarios for streaming (WireMock can return chunked text/event-stream), timeouts via withFixedDelay, and schema validation with JSON match operators. The moment your LLM integration grows beyond a single call, these stubs become the safety net that lets you refactor without fear. Capture the stubs in a src/test/resources/mappings folder to share them across teams, and consider a @WireMockTest annotation if you adopt the WireMock JUnit 5 extension for less boilerplate.

Tagsjavatestingwiremockllm-api

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
API integrationBuilding an async LLM client in Java with CompletableFutureBuild a non-blocking Java LLM client with CompletableFuture and HttpClient. Step-by-step async patterns for retries, fan-out, and timeouts.API integrationConnection pooling for high-throughput LLM calls in JavaPractical guide to java connection pooling llm api calls: configure Apache HttpClient or OkHttp for high throughput, avoid socket leaks, and tune pools.API integrationHandling rate limits in Java LLM API clientsLearn how to build resilient Java LLM API clients for java rate limit llm api scenarios using retry, backoff, and fallback patterns in production.API integrationJava virtual threads for concurrent LLM API callsPractical guide to Java virtual threads for concurrent LLM API calls: executor setup, structured concurrency, pinning pitfalls, and rate-limit tradeoffs.API integrationLangChain4j vs raw HTTP calls for Java LLM integrationA pragmatic head-to-head comparison of LangChain4j vs raw HTTP Java for LLM integration across capabilities, cost, latency, ergonomics, and limits.API integrationMaven setup for an OpenAI-compatible Java clientSet up a maven openai compatible java client with this step-by-step guide: dependencies, base URL config, sample code, and verification.

All java llm api integration posts →