Compiling a Rust LLM client to WASM

Shipping a rust llm client wasm target lets you run inference orchestration directly in the browser without a backend proxy. This guide walks through a minimal but production-shaped implementation: HTTP transport that works under wasm, proper async handling, and a small JS binding layer.

Step 1: Install the Rust WASM toolchain

Add the wasm32-unknown-unknown target and the wasm-pack builder. These are the only global prerequisites; the rest is per-crate.

rustup target add wasm32-unknown-unknown
cargo install wasm-pack

Verify the target is present:

rustup target list --installed | grep wasm32

You should see wasm32-unknown-unknown. If you already use wasm-bindgen for other projects, ensure wasm-pack is at least v0.12 — older versions emit deprecated JS glue.

Step 2: Scaffold the crate and pin dependencies

Create a library crate. A rust llm client wasm build must output a cdylib so wasm-pack can emit ES modules.

cargo new --lib rust-llm-wasm
cd rust-llm-wasm

Edit Cargo.toml to disable default features on reqwest and pull in the browser fetch backend. The wasm-client feature replaces the native TLS stack with fetch.

[package]
name = "rust-llm-wasm"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib", "rlib"]

[dependencies]
reqwest = { version = "0.11", default-features = false, features = ["json", "wasm-client"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
serde-wasm-bindgen = "0.6"
wasm-bindgen = "0.2"
wasm-bindgen-futures = "0.4"
anyhow = "1.0"

Run cargo check to confirm the dependency graph resolves. A common failure is pulling tokio with native features transitively; the default-features = false on reqwest prevents that.

Step 3: Model the OpenAI-compatible request

We target the OpenAI Chat Completions shape because every OpenAI-compatible gateway speaks it. Define the structs with serde.

use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct ChatMessage {
    pub role: String,
    pub content: String,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct ChatRequest {
    pub model: String,
    pub messages: Vec<ChatMessage>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub temperature: Option<f32>,
}

#[derive(Deserialize, Debug)]
pub struct ChatResponse {
    pub choices: Vec<Choice>,
}

#[derive(Deserialize, Debug)]
pub struct Choice {
    pub message: ChatMessage,
}

Keep the structs Clone if you plan to share the client across JS calls. The temperature field is optional so the JS side can omit it.

Step 4: Implement the client with browser fetch

reqwest::Client::new() works in wasm — it delegates to fetch and respects CORS. Build the URL, attach a bearer token, and serialize the body.

use reqwest::Client;
use anyhow::Result;

#[derive(Clone)]
pub struct LLMClient {
    base_url: String,
    api_key: String,
    http: Client,
}

impl LLMClient {
    pub fn new(base_url: impl Into<String>, api_key: impl Into<String>) -> Self {
        Self {
            base_url: base_url.into(),
            api_key: api_key.into(),
            http: Client::new(),
        }
    }

    pub async fn chat(&self, req: ChatRequest) -> Result<ChatResponse> {
        let url = format!("{}/chat/completions", self.base_url.trim_end_matches('/'));
        let resp = self.http
            .post(url)
            .bearer_auth(&self.api_key)
            .json(&req)
            .send()
            .await?
            .error_for_status()?
            .json::<ChatResponse>()
            .await?;
        Ok(resp)
    }
}

Point the base_url at any OpenAI-compatible endpoint. For multi-provider routing with automatic fallback when a provider is degraded, n4n.ai exposes one OpenAI-compatible endpoint covering 240+ models and forwards cache-control hints, but the request code above stays identical.

Verify the crate compiles for the host target first: cargo build. This catches type errors before wasm linking obscures them.

Step 5: Expose the client to JavaScript

wasm-bindgen cannot directly return Rust futures to JS. Wrap the client and convert the async call into a js_sys::Promise.

use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::future_to_promise;
use js_sys::Promise;

#[wasm_bindgen]
pub struct WasmLLMClient {
    inner: LLMClient,
}

#[wasm_bindgen]
impl WasmLLMClient {
    #[wasm_bindgen(constructor)]
    pub fn new(base_url: String, api_key: String) -> WasmLLMClient {
        WasmLLMClient { inner: LLMClient::new(base_url, api_key) }
    }

    #[wasm_bindgen]
    pub fn chat(&self, req_json: JsValue) -> Promise {
        let client = self.inner.clone();
        let fut = async move {
            let req: ChatRequest = serde_wasm_bindgen::from_value(req_json)
                .map_err(|e| JsValue::from_str(&e.to_string()))?;
            let resp = client.chat(req).await
                .map_err(|e| JsValue::from_str(&e.to_string()))?;
            serde_wasm_bindgen::to_value(&resp)
                .map_err(|e| JsValue::from_str(&e.to_string()))
        };
        future_to_promise(fut)
    }
}

The chat method takes a JsValue so callers pass a plain JS object. Errors become rejected promises with a string message — enough for browser logging.

Step 6: Build with wasm-pack

Compile to a web-targeted ES module. The --target web flag emits an ES module with no Node.js shims.

wasm-pack build --target web --out-dir pkg

Success produces pkg/rust_llm_wasm.js, pkg/rust_llm_wasm_bg.wasm, and TypeScript declarations. If the build fails with “cannot find crate for std”, the wasm target was not installed (see Step 1).

Step 7: Integrate into a web page

Create an index.html and a main.js (or .ts) that imports the generated module. Serve over HTTP — file:// will block wasm fetch.

<!doctype html>
<html>
<body>
  <script type="module">
    import { WasmLLMClient } from './pkg/rust_llm_wasm.js';

    const client = new WasmLLMClient(
      'https://api.openai.com/v1',
      'sk-your-key'
    );

    const req = {
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: 'Say hello in Rust.' }],
    };

    client.chat(req)
      .then(resp => console.log(resp.choices[0].message.content))
      .catch(err => console.error('LLM call failed:', err));
  </script>
</body>
</html>

Run a static server (python3 -m http.server) and open the page. The browser console should print the model’s text. If you see a CORS error, the endpoint is rejecting the browser’s preflight — OpenAI and compliant gateways send Access-Control-Allow-Origin: * for the completions route.

Step 8: Handle streaming and timeouts

The base rust llm client wasm above does a single blocking request. For production, add a timeout and consider streaming. reqwest wasm does not support timeout() (no timer API in browser fetch), so implement an AbortController via web-sys if you need cancellation. For streaming, use bytes_stream() and pipe chunks to a JS callback through wasm-bindgen closures.

A minimal timeout wrapper using wasm-bindgen-futures::spawn_local and js_sys::Promise::all with a setTimeout promise is about 30 lines; skip it until you observe hangs.

Verifying end-to-end

After Step 7, the definitive check is a successful console log from a real request. For CI, run wasm-pack test --chrome with a headless browser and a mock endpoint that returns a fixed ChatResponse JSON. That confirms the serialization round-trip and binding layer without burning API credits.

If you change the request schema, bump the crate version and rebuild — wasm-pack does not hot-reload. The JS object shape must match ChatRequest exactly, or from_value throws.