OkHttp vs java.net.http for calling LLM APIs in Java

Choosing an HTTP client shapes how you handle streaming tokens, retries, and timeouts when building against language model endpoints. The debate of okhttp vs java.net.http llm api integration in Java comes down to control versus zero-dependency convenience, not raw speed alone.

Capabilities

Both clients speak HTTP/1.1 and HTTP/2, but they expose different primitives for the patterns LLM APIs actually use: server-sent events, long timeouts, and per-request cancellation.

Streaming and SSE

LLM completions are typically streamed as SSE. java.net.http.HttpClient gives you a BodyHandlers.ofInputStream() or a custom BodySubscriber where you parse lines yourself. OkHttp exposes ResponseBody.source() which is a buffered BufferedSource — easier to read lines without blocking a single thread.

// java.net.http streaming
HttpClient jdk = HttpClient.newHttpClient();
HttpRequest req = HttpRequest.newBuilder()
    .uri(URI.create("https://api.example.com/v1/chat/completions"))
    .header("Content-Type", "application/json")
    .header("Authorization", "Bearer " + key)
    .POST(HttpRequest.BodyPublishers.ofString(json))
    .build();
jdk.send(req, HttpResponse.BodyHandlers.ofInputStream()).body()
    .transferTo(System.out); // naive; real code parses SSE

// OkHttp streaming
OkHttpClient ok = new OkHttpClient();
Request oreq = new Request.Builder()
    .url("https://api.example.com/v1/chat/completions")
    .addHeader("Authorization", "Bearer " + key)
    .post(RequestBody.create(json, MediaType.parse("application/json")))
    .build();
try (Response resp = ok.newCall(oreq).execute()) {
    BufferedSource src = resp.body().source();
    while (!src.exhausted()) {
        String line = src.readUtf8Line();
        // parse SSE data:
    }
}

If you route through a gateway like n4n.ai that automatically falls back across providers and honors cache-control hints, both clients behave identically as long as you forward the Cache-Control header and avoid prematurely closing the stream.

Timeouts and cancellation

java.net.http ties timeout to HttpRequest.timeout() (per request) and the client has no global read timeout — you must wrap sendAsync in orTimeout or use a BodySubscriber that enforces deadlines. OkHttp has explicit call.timeout(), connectTimeout, readTimeout, and writeTimeout at client or call level.

Async and backpressure

java.net.http returns CompletableFuture<HttpResponse<T>> from sendAsync. Cancellation propagates via future.cancel(true), which aborts the underlying connection. OkHttp uses Call.enqueue(Callback) and Call.cancel(); backpressure is manual since bytes arrive into Okio buffers.

When evaluating okhttp vs java.net.http llm api streaming, the buffered source and explicit cancellation in OkHttp reduce boilerplate for token-by-token UIs.

Price and Cost Model

Neither library charges a fee. java.net.http ships inside OpenJDK under the GPLv2+Classpath exception; OkHttp is Apache 2.0. The real cost is dependency footprint and maintenance surface.

OkHttp pulls in Okio and (in recent versions) a small Kotlin stdlib slice — roughly 1.5 MB of jars. java.net.http is already in the JRE since Java 11; zero additional artifacts. For a thin container or GraalVM native image, the JDK client reduces reflection config and resource hint work. OkHttp works on GraalVM but needs reflect-config entries for its connection pool and DNS.

Dependency risk is inverted: JDK bugs get fixed in your base image upgrade; OkHttp bugs require a library bump and rebuild. For regulated teams, the JDK path simplifies supply-chain review.

Latency and Throughput

Synthetic microbenchmarks vary, but in practice both saturate a 1 Gbps link with similar throughput. The difference is in connection reuse and TLS overhead.

Connection pooling

java.net.http maintains a per-client pool with HTTP/2 multiplexing; you cannot tune idle socket lifetimes. OkHttp lets you set connectionPool(new ConnectionPool(100, 5, TimeUnit.MINUTES)) and pingInterval for HTTP/2 keepalive. For high-concurrency LLM proxy loops issuing thousands of small prompts, OkHttp’s tunables avoid stale socket resets behind NAT.

Warmup and GC

java.net.http relies on java.nio channels and direct buffers; OkHttp uses Okio segments. Neither dominates p99. Expect model inference time — not client serialization — to dominate tail latency. TLS session resumption works in both; JDK uses the default SSLContext, OkHttp lets you inject a custom one for mTLS to internal gateways.

Ergonomics

Request building and JSON

java.net.http is verbose: BodyPublishers.ofString plus manual ObjectMapper calls. OkHttp’s Request.Builder is fluent and pairs well with Retrofit or Moshi.

// OkHttp with interceptor for logging
OkHttpClient client = new OkHttpClient.Builder()
    .addInterceptor(chain -> {
        Request r = chain.request();
        System.out.println("-> " + r.url());
        return chain.proceed(r);
    }).build();

Both require an external JSON lib; neither bundles one.

Error handling

java.net.http throws HttpTimeoutException or IOException; status code is in HttpResponse.statusCode(). OkHttp separates network failures (IOException) from application errors (response.code()), and centralizes mapping via interceptors. For LLM APIs that return 429 with Retry-After, OkHttp’s interceptor can sleep-and-retry without scattering logic at call sites.

Ecosystem

OkHttp has interceptors, event listeners, and first-class mocking via MockWebServer. java.net.http has no interceptor API; you fake endpoints with WireMock or a local com.sun.net.httpserver. For observability, OkHttp’s EventListener reports connectAcquired, requestHeadersStart, and responseBodyEnd timestamps; the JDK client requires manual instrumentation around sendAsync.

Retrofit and Feign both target OkHttp. If you already use those, adding java.net.http means writing a custom Call.Factory — possible but unpopular.

Limits

java.net.http requires Java 11+. On Android, it is absent; Android apps use HttpURLConnection or OkHttp. OkHttp supports Android API 21+ and older JVMs. JPMS: java.net.http is a clean module (java.net.http); OkHttp is an automatic module in older builds, named okhttp3 in recent ones.

Neither client implements the OpenAI streaming protocol for you — both hand you bytes. If you need typed SSE events, add eventsource (OkHttp) or a small parser for JDK.

Comparison Table

Which to Choose

Greenfield backend on Java 17+

Use java.net.http. Zero dependencies, built-in HTTP/2, and sufficient for straightforward LLM calls. Wrap SSE parsing in a small reusable helper.

Android or mixed JVM/Android

OkHttp. It is the de facto standard and avoids compatibility shims.

High-control proxy or gateway with retries

OkHttp. Interceptors let you inject fallback logic, sign requests, and meter tokens without polluting business code.

Quick script or CLI tool

java.net.http keeps the artifact count at zero beyond the JDK. No build file needed beyond a javac invocation.

The okhttp vs java.net.http llm api decision is less about throughput and more about how much control you want over the HTTP layer. Pick the one that matches your deployment target, timeout requirements, and existing ecosystem.