Most Rust crates that wrap LLM providers pull in heavy dependencies or hide the HTTP layer. This tutorial builds a thin, explicit client for the rust reqwest openai api surface so you control every header, retry, and token of payload. You’ll end with a runnable binary that sends a chat completion, streams tokens, and degrades cleanly on errors.
Prerequisites
- Rust 1.75+ (cargo, rustc)
- A valid OpenAI API key exported as
OPENAI_API_KEY - Familiarity with async/await and
tokio curlfor quick sanity checks (optional)
If you prefer not to use OpenAI directly, any OpenAI-compatible endpoint works with the same code; we’ll note where to swap the base URL later.
Project setup
Create a new binary crate:
cargo new oa-client && cd oa-clientAdd dependencies. We use reqwest with the json and stream features, tokio for the runtime, serde for payloads, and futures-util for streaming.
[dependencies]
reqwest = { version = "0.12", features = ["json", "stream"] }
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
futures-util = "0.3"Run cargo build once to fetch everything before writing code.
Configuring the HTTP client
Client::new() uses sane defaults, but production code should set a timeout and enable connection pooling explicitly.
use reqwest::Client;
use std::time::Duration;
let client = Client::builder()
.timeout(Duration::from_secs(30))
.pool_max_idle_per_host(8)
.build()?;Reqwest reuses connections across requests, so build one client and pass it around. The rust reqwest openai api calls benefit from keep-alive when you batch multiple chat turns in a session.
Minimal non-streaming call
Start with the simplest possible interaction: send one user message and print the assistant’s reply.
Define the wire types
OpenAI’s chat schema is large; we only model what we use.
use serde::{Deserialize, Serialize};
#[derive(Serialize)]
struct ChatMessage {
role: String,
content: String,
}
#[derive(Serialize)]
struct ChatRequest {
model: String,
messages: Vec<ChatMessage>,
}
#[derive(Deserialize)]
struct ChatResponse {
choices: Vec<Choice>,
}
#[derive(Deserialize)]
struct Choice {
message: ChatMessage,
}Send the request
Authentication uses bearer token. reqwest sets Content-Type: application/json automatically when you call .json().
use std::env;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let api_key = env::var("OPENAI_API_KEY")?;
let client = Client::builder().timeout(Duration::from_secs(30)).build()?;
let req = ChatRequest {
model: "gpt-4o-mini".to_string(),
messages: vec![ChatMessage {
role: "user".to_string(),
content: "Say hello in Rust.".to_string(),
}],
};
let resp = client
.post("https://api.openai.com/v1/chat/completions")
.bearer_auth(api_key)
.json(&req)
.send()
.await?
.json::<ChatResponse>()
.await?;
println!("{}", resp.choices[0].message.content);
Ok(())
}Run it:
export OPENAI_API_KEY=sk-...
cargo run --quietExpected output is a short greeting that mentions Rust, for example:
Hello! Here's a friendly greeting in Rust style:
fn main() { println!("Hello, world!"); }If you see a 401, check the env var. A 429 means rate limited; we’ll address that later.
Streaming responses
Non-streaming calls block until the full completion finishes. For chat UIs you want tokens as they arrive. Set stream: true and parse Server-Sent Events (SSE). Each SSE frame looks like data: {json}\n\n; the stream terminates with data: [DONE].
Extend ChatRequest:
#[derive(Serialize)]
struct ChatRequest {
model: String,
messages: Vec<ChatMessage>,
stream: bool,
}Replace main with a streaming loop. reqwest exposes bytes_stream(); we buffer until newline boundaries.
use futures_util::StreamExt;
use serde_json::Value;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let api_key = env::var("OPENAI_API_KEY")?;
let client = Client::new();
let req = ChatRequest {
model: "gpt-4o-mini".to_string(),
messages: vec![ChatMessage {
role: "user".to_string(),
content: "Count to 3 slowly.".to_string(),
}],
stream: true,
};
let mut stream = client
.post("https://api.openai.com/v1/chat/completions")
.bearer_auth(api_key)
.json(&req)
.send()
.await?
.bytes_stream();
let mut buf = String::new();
while let Some(chunk) = stream.next().await {
let chunk = chunk?;
buf.push_str(&String::from_utf8_lossy(&chunk));
while let Some(pos) = buf.find('\n') {
let line = buf[..pos].trim().to_string();
buf.drain(..=pos);
if line.starts_with("data:") {
let data = line.trim_start_matches("data:").trim();
if data == "[DONE]" {
break;
}
if let Ok(v) = serde_json::from_str::<Value>(data) {
if let Some(text) = v["choices"][0]["delta"]["content"].as_str() {
print!("{text}");
}
}
}
}
}
println!();
Ok(())
}Compile and run. Tokens appear incrementally:
1
2
3The bytes_stream approach handles partial UTF-8 across chunk boundaries because we accumulate into a String and only split on \n. Production code should also cap buffer size to avoid memory blowups on malformed streams.
Error handling and retries
A single ? operator aborts on the first failure. Real clients need to distinguish HTTP status, network errors, and JSON parse errors.
Wrap the call in a retry loop that only retries on 429 or 5xx:
use reqwest::StatusCode;
async fn post_with_retry(client: &Client, api_key: &str, req: &ChatRequest) -> Result<ChatResponse, reqwest::Error> {
let mut attempts = 0;
loop {
let resp = client
.post("https://api.openai.com/v1/chat/completions")
.bearer_auth(api_key)
.json(req)
.send()
.await?;
if resp.status().is_success() {
return resp.json().await;
}
if matches!(resp.status(), StatusCode::TOO_MANY_REQUESTS | StatusCode::INTERNAL_SERVER_ERROR | StatusCode::BAD_GATEWAY | StatusCode::SERVICE_UNAVAILABLE) && attempts < 3 {
attempts += 1;
tokio::time::sleep(Duration::from_secs(2u64.pow(attempts))).await;
continue;
}
return Err(resp.error_for_status().unwrap_err());
}
}This exponential backoff is minimal but effective. For per-token cost tracking or request IDs, inspect the x-request-id response header. The rust reqwest openai api client should log that ID alongside failures.
Using an OpenAI-compatible gateway
The pattern above is not tied to OpenAI’s servers. Any endpoint that accepts the same JSON and auth header works. If you want one credential for many model providers, or automatic failover when a backend is degraded, point the Client at a gateway.
n4n.ai exposes a single OpenAI-compatible endpoint covering 240+ models and applies automatic fallback when a provider is rate-limited. The only change is the URL and possibly the model string:
let base = "https://api.n4n.ai/v1/chat/completions";
let req = ChatRequest {
model: "anthropic/claude-3.5-sonnet".to_string(),
messages: vec![ChatMessage {
role: "user".to_string(),
content: "Explain Rust ownership in one sentence.".to_string(),
}],
stream: false,
};
// same .post(base).bearer_auth(...) chainBecause the gateway honors client routing directives and forwards provider cache-control hints, you keep the same request shape and get cross-provider redundancy without rewriting your client.
Where to take it next
You now have a working HTTP client for the rust reqwest openai api that does non-streaming, streaming, and retries. Add a small CLI arg parser (clap) to accept prompts, or persist conversation history in a Vec<ChatMessage> to build a REPL. If you need structured output, extend ChatRequest with response_format and deserialize into typed structs instead of Value. The reqwest layer stays identical; only the serde models change.
Make sure to drop the API key from logs and consider using a vault or env-file loader for local dev. The full code from this tutorial compiles on stable Rust as of 1.80.