How to proxy OpenAI-compatible requests through Express

A well-designed express.js proxy openai-compatible api endpoint lets you hide provider keys, enforce rate limits, and log traffic without touching client SDK calls. The OpenAI wire protocol is just HTTP POST with JSON or SSE, so Express can forward it transparently. Below is a complete, runnable path from empty directory to a streaming proxy you can point the official SDK at.

Step 1: Scaffold the project and install dependencies

Create a directory and install the only three runtime packages you need: Express as the HTTP layer, http-proxy-middleware to forward streams without buffering, and cors if browsers will call the proxy directly.

mkdir llm-proxy && cd llm-proxy
npm init -y
npm install express http-proxy-middleware cors dotenv

dotenv keeps your provider key out of source control. http-proxy-middleware wraps http-proxy, which pipes raw request and response streams—critical for token-by-token LLM output.

Step 2: Stand up a minimal Express server

Start with a skeleton that listens and exposes a health check. Do not mount express.json() globally. Any body parser consumes the request stream, which breaks streaming proxies.

// server.js
require('dotenv').config();
const express = require('express');
const cors = require('cors');

const app = express();
app.use(cors());

app.get('/health', (req, res) => res.json({ ok: true }));

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`Proxy listening on ${PORT}`));

Run node server.js. curl localhost:3000/health should return {"ok":true}.

Step 3: Mount the proxy route

The core of any express.js proxy openai-compatible api is a single route that forwards /v1/* to the upstream base URL. OpenAI and most gateways expose the same path prefix, so a pass-through is trivial.

const { createProxyMiddleware } = require('http-proxy-middleware');

const TARGET = process.env.TARGET_BASE_URL || 'https://api.openai.com';

app.use('/v1', createProxyMiddleware({
  target: TARGET,
  changeOrigin: true,
  pathRewrite: { '^/v1': '/v1' }, // explicit no-op for clarity
  logger: console,
}));

changeOrigin rewrites the Host header to match the target, which many providers require. If your target is a gateway such as n4n.ai, the same proxy works; the gateway honors client routing directives and provides fallback, but your Express layer stays dumb.

Step 4: Inject server-side credentials

Never trust client-supplied authorization. Strip incoming Authorization headers and set your own from environment variables. This is the primary security win of the proxy.

app.use('/v1', createProxyMiddleware({
  target: TARGET,
  changeOrigin: true,
  onProxyReq: (proxyReq, req, res) => {
    proxyReq.removeHeader('authorization');
    proxyReq.removeHeader('Authorization');
    if (!process.env.PROVIDER_API_KEY) {
      res.status(500).json({ error: 'missing_provider_key' });
      return;
    }
    proxyReq.setHeader('Authorization', `Bearer ${process.env.PROVIDER_API_KEY}`);
  },
  onError: (err, req, res) => {
    res.status(502).json({ error: 'proxy_error', message: err.message });
  },
}));

If you need to forward custom routing hints (e.g., x-provider-preference), set them in onProxyReq the same way.

Step 5: Preserve streaming and handle errors

http-proxy-middleware streams by default. The only way you break SSE is by adding a body parser or manually reading req. Keep the middleware order clean: logging first, then proxy.

Add a timeout so hung upstream connections don’t pin Node event loop resources:

app.use('/v1', createProxyMiddleware({
  target: TARGET,
  changeOrigin: true,
  proxyTimeout: 120000,
  timeout: 120000,
  onProxyReq: (proxyReq, req, res) => {
    proxyReq.removeHeader('authorization');
    proxyReq.setHeader('Authorization', `Bearer ${process.env.PROVIDER_API_KEY}`);
  },
  onError: (err, req, res) => {
    if (!res.headersSent) {
      res.status(502).json({ error: 'proxy_error', message: err.message });
    }
  },
}));

Step 6: Add request logging

A simple logging middleware mounted before the proxy gives you audit trails without external dependencies.

app.use('/v1', (req, res, next) => {
  const start = Date.now();
  res.on('finish', () => {
    console.log(`${req.method} ${req.originalUrl} -> ${res.statusCode} (${Date.now() - start}ms)`);
  });
  next();
});

Place this app.use above the proxy app.use so it wraps the request.

Step 7: Point your client at the proxy

The OpenAI Node SDK accepts a baseURL. Point it at your local Express port. The SDK will send requests to /v1/chat/completions; the proxy forwards them upstream with the real key.

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:3000/v1',
  apiKey: 'unused', // proxy injects the real key
});

const completion = await client.chat.completions.create({
  model: 'gpt-3.5-turbo',
  messages: [{ role: 'user', content: 'Say hello.' }],
  stream: true,
});

for await (const chunk of completion) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}

A raw curl proves the wire format is intact:

curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"hi"}]}'

Step 8: Verify streaming success

Run the streaming SDK script or the equivalent curl with -N:

curl -N -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"hi"}],"stream":true}'

You should see lines beginning with data: and a final data: [DONE]. If you see the full JSON blob instead, the proxy buffered the response—check for stray express.json() or similar middleware in the route chain. Server logs should show POST /v1/chat/completions -> 200.

Step 9: Hardening for production

The express.js proxy openai-compatible api you have now is functional but bare. Before production, add:

• Rate limiting with express-rate-limit on /v1 to protect upstream quotas.
• Per-token metering if you resell access; read res body size or use the x-usage headers some gateways return.
• Health checks that hit TARGET/health or a lightweight model call.
• TLS termination via a reverse proxy (nginx, Caddy) or Node https with certs.

Example rate limit:

const rateLimit = require('express-rate-limit');
const limiter = rateLimit({ windowMs: 60_000, max: 60 });
app.use('/v1', limiter);

Mount it before the proxy. If the upstream supports provider cache-control hints, your proxy already forwards them untouched—no extra code needed.

Step 10: Clean shutdown

Node will exit with open sockets if you don’t handle signals. Add:

process.on('SIGTERM', () => {
  console.log('SIGTERM received, closing');
  process.exit(0);
});

That’s the full loop: scaffold, mount, inject auth, stream, verify, harden. The client code stays 100% OpenAI-compatible, and you control the policy layer.