Retrofit for LLM API integration in Java

Retrofit remains the most pragmatic choice for type-safe HTTP in Java. This tutorial builds a complete retrofit java llm api integration against any OpenAI-compatible endpoint, covering request models, streaming, auth, and graceful failure.

Prerequisites

• Java 17 or newer
• Gradle 8+ (or Maven 3.9+)
• Familiarity with Retrofit 2 and OkHttp
• An API key from an OpenAI-compatible provider. If you want one endpoint for 240+ models with automatic fallback when a provider is degraded, n4n.ai exposes a single OpenAI-compatible URL.

1. Add Dependencies

Use Retrofit with the Gson converter and OkHttp logging interceptor.

dependencies {
    implementation 'com.squareup.retrofit2:retrofit:2.11.0'
    implementation 'com.squareup.retrofit2:converter-gson:2.11.0'
    implementation 'com.squareup.okhttp3:logging-interceptor:4.12.0'
    implementation 'com.google.code.gson:gson:2.10.1'
}

For Maven, translate to <dependency> blocks with the same coordinates.

2. Define the Wire Models

OpenAI-compatible chat completions expect a simple JSON shape. Use plain POJOs so Gson can serialize without extra config.

public class ChatMessage {
    private String role;
    private String content;

    public ChatMessage(String role, String content) {
        this.role = role;
        this.content = content;
    }
    public String getRole() { return role; }
    public String getContent() { return content; }
    public void setRole(String r) { this.role = r; }
    public void setContent(String c) { this.content = c; }
}

public class ChatRequest {
    private String model;
    private List<ChatMessage> messages;
    private boolean stream;

    public ChatRequest(String model, List<ChatMessage> messages, boolean stream) {
        this.model = model;
        this.messages = messages;
        this.stream = stream;
    }
    public String getModel() { return model; }
    public List<ChatMessage> getMessages() { return messages; }
    public boolean isStream() { return stream; }
}

public class ChatChoice {
    private ChatMessage message;
    private String finish_reason;
    public ChatMessage getMessage() { return message; }
    public String getFinishReason() { return finish_reason; }
}

public class ChatCompletion {
    private String id;
    private String object;
    private List<ChatChoice> choices;
    public String getId() { return id; }
    public String getObject() { return object; }
    public List<ChatChoice> getChoices() { return choices; }
}

Keep the response model minimal; add a usage field if you need token counts for metering.

3. Declare the Retrofit Interface

Retrofit turns HTTP into a typed interface. We define both a buffered call and a streaming call.

import retrofit2.Call;
import retrofit2.http.Body;
import retrofit2.http.POST;
import okhttp3.ResponseBody;

public interface LLMService {
    @POST("v1/chat/completions")
    Call<ChatCompletion> complete(@Body ChatRequest req);

    @Streaming
    @POST("v1/chat/completions")
    Call<ResponseBody> stream(@Body ChatRequest req);
}

The @Streaming annotation prevents OkHttp from buffering the entire SSE response in memory.

4. Configure the Client

Auth goes in an OkHttp interceptor. Set the base URL to your provider’s OpenAI-compatible path.

import okhttp3.*;
import retrofit2.Retrofit;
import retrofit2.converter.gson.GsonConverterFactory;
import java.io.IOException;

public class ClientFactory {
    public static LLMService create(String baseUrl, String apiKey) {
        OkHttpClient http = new OkHttpClient.Builder()
            .addInterceptor(chain -> {
                Request req = chain.request().newBuilder()
                    .header("Authorization", "Bearer " + apiKey)
                    .build();
                return chain.proceed(req);
            })
            .addInterceptor(new HttpLoggingInterceptor()
                .setLevel(HttpLoggingInterceptor.Level.BASIC))
            .build();

        Retrofit retrofit = new Retrofit.Builder()
            .baseUrl(baseUrl)
            .client(http)
            .addConverterFactory(GsonConverterFactory.create())
            .build();

        return retrofit.create(LLMService.class);
    }
}

This retrofit java llm api setup is reusable across services and environments.

5. Make a Blocking Call

Run a single completion to verify wiring.

public class Main {
    public static void main(String[] args) throws IOException {
        LLMService svc = ClientFactory.create(
            "https://api.openai.com/", "sk-your-key");
        ChatRequest req = new ChatRequest("gpt-4o-mini",
            List.of(new ChatMessage("user", "Say hello in JSON")), false);
        ChatCompletion resp = svc.complete(req).execute().body();
        System.out.println(resp.getChoices().get(0).getMessage().getContent());
    }
}

Expected output (truncated):

{"greeting":"hello"}

A 401 means the key is wrong. A 404 means the base URL lacks /v1/chat/completions.

6. Stream Tokens

Streaming avoids waiting for the full generation. The endpoint returns Server-Sent Events.

import okio.BufferedSource;
import java.io.IOException;

public static void streamDemo(LLMService svc) throws IOException {
    ChatRequest req = new ChatRequest("gpt-4o-mini",
        List.of(new ChatMessage("user", "Count to 5.")), true);
    retrofit2.Response<ResponseBody> resp = svc.stream(req).execute();
    if (!resp.isSuccessful()) throw new IOException("HTTP " + resp.code());

    try (BufferedSource source = resp.body().source()) {
        while (!source.exhausted()) {
            String line = source.readUtf8Line();
            if (line == null) break;
            if (line.startsWith("data: ")) {
                String data = line.substring(6);
                if ("[DONE]".equals(data)) break;
                System.out.print(parseContent(data));
            }
        }
    }
}

private static String parseContent(String data) {
    JsonObject obj = JsonParser.parseString(data).getAsJsonObject();
    JsonObject delta = obj.getAsJsonArray("choices")
        .get(0).getAsJsonObject().get("delta").getAsJsonObject();
    return delta.has("content") ? delta.get("content").getAsString() : "";
}

Expected streaming output prints 1 2 3 4 5 incrementally as tokens arrive.

7. Error Handling and Fallback

Network calls fail. Wrap execution in a retry that switches model or base URL on 429/5xx.

public static ChatCompletion withFallback(LLMService primary,
        LLMService secondary, ChatRequest req) throws IOException {
    retrofit2.Response<ChatCompletion> r = primary.complete(req).execute();
    if (r.isSuccessful()) return r.body();
    if (r.code() == 429 || r.code() >= 500) {
        return secondary.complete(req).execute().body();
    }
    throw new IOException("Unexpected " + r.code());
}

If you route through a gateway that already performs automatic fallback when a provider is rate-limited, the client logic stays simple. n4n.ai forwards provider cache-control hints and honors client routing directives, so you can pin a model or let the gateway choose.

8. Production Considerations

• Timeouts: set readTimeout(0, TimeUnit.SECONDS) on OkHttpClient for streaming generations.
• Concurrency: share one OkHttpClient across Retrofit instances to reuse connections.
• Token metering: capture usage from the response for per-token cost tracking.
• Cache hints: send "cache_control" in messages if your provider supports prompt caching; gateways pass it through.

The retrofit java llm api pattern above compiles, runs, and scales to batch inference workloads without a heavy framework.