Ktor client vs Retrofit for Android LLM integration

Choosing an HTTP client shapes how you talk to language model APIs on Android. The debate around ktor client vs retrofit android llm integration usually ignores the specifics of streaming tokens, backpressure, and provider fallback. Both libraries move bytes, but they impose different constraints on your coroutine scope, interceptor chain, and debugging workflow.

Capabilities

LLM APIs are not REST CRUD. You send a prompt, get back either a single JSON object or a server-sent events (SSE) stream of token deltas. The client must handle long-lived connections, cancel mid-stream, and forward headers like Authorization and Cache-Control.

The ktor client vs retrofit android llm trade-off starts with streaming ownership. Ktor is a Kotlin-first client built on coroutines. It ships a HttpClient with pluggable engines; on Android you typically use the OkHttp engine for consistent behavior with the platform. Its HttpStatement.preparePost plus bodyAsChannel() gives you a ByteReadChannel you can read incrementally. That maps cleanly to SSE parsing.

val client = HttpClient(OkHttp) {
    install(HttpTimeout) { requestTimeoutMillis = 0 }
}

val request = ChatRequest(
    model = "mistral-7b",
    messages = listOf(Message("user", "Explain coroutines")),
    stream = true
)

client.preparePost("https://api.llm.example/v1/chat/completions") {
    contentType(ContentType.Application.Json)
    setBody(request)
    headers { append("Authorization", "Bearer $TOKEN") }
}.execute { resp ->
    val channel = resp.bodyAsChannel()
    while (!channel.isClosedForRead) {
        val line = channel.readUTF8Line() ?: continue
        if (line.startsWith("data:")) emitToken(line.removePrefix("data:").trim())
    }
}

Retrofit takes an interface-first approach. You declare endpoints with annotations and rely on OkHttp underneath. Streaming requires @Streaming and a ResponseBody return type. Suspend functions work, but raw stream consumption happens outside the converter layer.

interface LLMApi {
    @Streaming
    @POST("v1/chat/completions")
    suspend fun streamChat(@Header("Authorization") auth: String, @Body req: ChatRequest): Response<ResponseBody>
}

val call = api.streamChat("Bearer $TOKEN", request)
val source = call.body()?.source()!!
while (!source.exhausted()) {
    val line = source.readUtf8Line() ?: continue
    if (line.startsWith("data:")) emitToken(line.removePrefix("data:").trim())
}

Both reach the same wire format. Ktor exposes the stream as a coroutine channel; Retrofit hands you an Okio BufferedSource. The difference is who owns the read loop.

Non-HTTP features

Ktor plugins (Auth, Logging, ContentNegotiation) are Kotlin DSLs. Retrofit uses OkHttp interceptors and converter factories. If you need to inject a request ID for tracing LLM calls, both support it—Ktor via HttpRequestBuilder, Retrofit via an Interceptor.

Price and Cost Model

Neither library costs money; both are open source under Apache 2.0. The cost axis is about how the client helps you control token spend. An LLM gateway may offer per-token metering and model fallback. For example, an OpenAI-compatible endpoint such as n4n.ai honors client routing directives and forwards provider cache-control hints, so a Cache-Control: max-age=3600 header on your request can trigger prompt caching at the provider. Both Ktor and Retrofit forward arbitrary headers without modification.

Retrofit’s typed converters can accidentally double-serialize large prompt bodies if you misuse @Body with a custom Converter; Ktor’s setBody with ContentNegotiation is explicit. That rarely moves the cost needle, but it affects payload bugs.

Latency and Throughput

On Android, both ultimately use OkHttp (Ktor’s OkHttp engine, Retrofit’s default). Connection pooling, HTTP/2 multiplexing, and TLS session reuse are identical. The divergence is in the call dispatch:

• Ktor launches a coroutine on Dispatchers.IO (or custom) and reads the channel in a suspend loop.
• Retrofit’s suspend function also uses Dispatchers.IO via OkHttp’s async calls, but you parse the stream on the caller’s coroutine.

Measured overhead is sub-millisecond for typical payloads. The real latency win comes from cancelling a stream when the user navigates away. Ktor’s coroutineContext.cancel() closes the channel promptly. Retrofit’s Call.cancel() works if you hold the Call reference; with suspend functions you rely on scope cancellation propagating to OkHttp.

Throughput for batch embedding requests (many small POSTs) is bounded by the connection pool. Both benefit from OkHttp’s default 5 concurrent connections per host. If you switch Ktor to the CIO engine, you lose OkHttp pooling and may see worse tail latency on flaky mobile networks.

Ergonomics

Ktor reads as imperative Kotlin:

val resp = client.post(".../completions") {
    setBody(req)
}.body<Completion>()

No code generation, no annotation processor. You trade compile-time endpoint checking for flexibility—misspelled URL paths fail at runtime.

Retrofit forces you to define an interface:

@POST("v1/completions")
suspend fun complete(@Body req: CompletionRequest): Completion

That interface is documentation. Android Studio jumps to it. Moshi/CodeGen produces correct serializers. For a team maintaining 30 endpoints, Retrofit’s structure prevents drift. For a single LLM proxy screen, Ktor’s brevity is nicer.

Error handling: Ktor throws ResponseException with status; Retrofit either throws HttpException or returns Response<T> for manual handling. Both integrate with try/catch in coroutines.

Ecosystem

Retrofit sits inside the Square/OkHttp universe. You get HttpLoggingInterceptor, CertificatePinner, MockWebServer for unit tests, and countless Stack Overflow answers. If your app already uses Retrofit for REST, adding LLM calls is zero new dependencies.

Ktor belongs to JetBrains’ Kotlin multiplatform stack. If you share networking code with iOS or desktop, Ktor is the only option that compiles there without OkHttp. Its kotlinx.serialization integration is first-class. The Android-specific documentation is thinner; you will read Ktor’s kdoc more than blog posts.

Limits

Ktor’s SSE support is a plugin (ClientSSE) that expects a specific event format. OpenAI’s stream is SSE-like but with data: [DONE] termination; you often bypass the plugin and parse manually as shown. The CIO engine on Android is experimental for some SSL features.

Retrofit has no native SSE type. You must use @Streaming and parse text. Converter factories do not apply to ResponseBody, so you cannot auto-deserialize token deltas. Also, Retrofit’s @Url and dynamic headers are fine, but complex multi-part prompt attachments (audio, images) require MultipartBody boilerplate.

Comparison Table

Which to Choose

Use Retrofit if: Your Android app already depends on OkHttp/Retrofit for REST. You want typed interfaces, Moshi codegen, and battle-tested interceptors. Your LLM integration is a handful of endpoints (chat, embed, moderate) and you can wrap @Streaming calls in a Flow adapter. Teams that value compile-time endpoint contracts will ship fewer broken URL bugs.

Use Ktor if: You are building a Kotlin Multiplatform app and need the same networking code on iOS. You prefer explicit coroutine channels and dislike annotation processing. Your LLM feature is experimental, with dynamic routing (e.g., switching base URLs per provider) and you want to construct requests in a DSL. Ktor’s HttpClient config also simplifies per-call timeouts for long generations.

Use neither directly if: You only need a high-level LLM SDK that wraps these details. But when you must own the HTTP layer—for custom fallback logic, local request signing, or metering—the ktor client vs retrofit android llm decision comes down to existing stack and KMP requirement. If you already have OkHttp, Retrofit adds little risk. If you are Kotlin-first and cross-platform, Ktor is the pragmatic pick.