Deploying an Express LLM backend with PM2

Building an express.js deploy pm2 llm backend is mostly about treating the model endpoint as a normal upstream API and letting PM2 own the process lifecycle. This guide takes you from an empty directory to a supervised, clustered Node service that proxies chat completions with proper timeouts and logging.

Step 1: Scaffold the project

Create a working directory and initialize a minimal Node package. Use ESM or CommonJS; the examples below use CommonJS for widest compatibility on older AMIs.

mkdir llm-gateway && cd llm-gateway
npm init -y
npm install express dotenv openai

Pin your Node version in package.json to whatever you tested with. PM2 will use the system Node, so keep it boring:

{
  "engines": {
    "node": ">=18.19 <21"
  }
}

Create a .env file (see Step 3) and a server.js entrypoint. Do not add a frontend or template engine—this is a backend service.

Step 2: Implement the LLM proxy route

The core job of the express.js deploy pm2 llm backend is to accept a request from your app, attach auth and routing hints, and forward it to the model provider. Use the official openai SDK with a custom baseURL so you can swap providers without touching route code.

// server.js
const express = require('express');
const dotenv = require('dotenv');
const OpenAI = require('openai');

dotenv.config();

const app = express();
app.use(express.json({ limit: '2mb' }));

const client = new OpenAI({
  apiKey: process.env.LLM_API_KEY,
  baseURL: process.env.LLM_BASE_URL,
  timeout: 30_000,
  maxRetries: 2,
});

app.post('/v1/chat/completions', async (req, res) => {
  const { model = 'gpt-4o-mini', messages, stream = false } = req.body;
  try {
    const completion = await client.chat.completions.create({
      model,
      messages,
      stream,
      // forward cache hint if client sent one
      headers: req.headers['cache-control']
        ? { 'cache-control': req.headers['cache-control'] }
        : undefined,
    });
    res.json(completion);
  } catch (err) {
    console.error('upstream error', err.status, err.message);
    res.status(err.status || 502).json({ error: err.message });
  }
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`listening on ${PORT}`));

If you want automatic fallback across providers, set LLM_BASE_URL to a gateway such as n4n.ai which exposes one OpenAI-compatible endpoint for 240+ models and handles degradation without extra code. The express.js deploy pm2 llm backend stays identical; only the env var changes.

Step 3: Environment and secrets

Never hardcode keys. Use .env locally and inject real values via PM2 env or systemd on the host.

# .env
PORT=3000
LLM_API_KEY=sk-xxxxxxxx
LLM_BASE_URL=https://api.openai.com/v1

Add .env to .gitignore. For production, export the vars before pm2 start or use PM2’s env block (Step 5). Rotate keys by restarting the process, not by editing code.

Step 4: Harden the server

A bare listen will leak stack traces and hang on slow upstreams. Add a global timeout and a catch-all error handler. PM2 will restart on uncaught exceptions, but you should still return clean 5xx.

// append to server.js
app.use((err, req, res, next) => {
  console.error('unhandled', err);
  res.status(500).json({ error: 'internal' });
});

// force-close lingering sockets after 35s
const server = app.listen(PORT, () => console.log(`listening on ${PORT}`));
server.timeout = 35_000;
server.keepAliveTimeout = 30_000;

If you expect streaming, handle req.on('close') to abort the upstream call. The OpenAI SDK supports signal via AbortController; wire it in before going live.

Step 5: PM2 ecosystem config

PM2 reads ecosystem.config.js to standardize launches. Use cluster mode to exploit multi-core hosts; Node’s single-threaded event loop otherwise wastes CPU.

// ecosystem.config.js
module.exports = {
  apps: [
    {
      name: 'llm-backend',
      script: 'server.js',
      instances: 'max',
      exec_mode: 'cluster',
      autorestart: true,
      max_memory_restart: '512M',
      env: {
        PORT: 3000,
        LLM_BASE_URL: 'https://api.openai.com/v1',
      },
      env_production: {
        PORT: 3000,
        LLM_BASE_URL: 'https://api.openai.com/v1',
        NODE_ENV: 'production',
      },
    },
  ],
};

Keep secrets out of this file. Load them from the shell: export LLM_API_KEY=... before start, or use pm2 set llm-backend:LLM_API_KEY "sk-...".

Step 6: Launch and monitor with PM2

Start the service in cluster mode and persist the process list so it survives reboots.

pm2 start ecosystem.config.js --env production
pm2 save
pm2 logs llm-backend

Check the process table:

pm2 status

You should see N instances (one per core) in online state. If a worker crashes, PM2 restarts it and increments restart count. Wire pm2 startup to generate a systemd unit so the gateway boots on machine restart.

For zero-downtime config changes, use pm2 reload llm-backend instead of restart—cluster mode rotates workers.

Step 7: Verify the deployment

Hit the local endpoint with a minimal chat payload. Success means a 200 with choices and no upstream errors in pm2 logs.

curl -s localhost:3000/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}' | head -c 200

Expected response fragment:

{"id":"chatcmpl-...","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","content":"pong"}]}

Next, validate concurrency. Run a quick loop or hey against the port:

hey -n 100 -c 10 -m POST -H 'content-type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"load test"}]}' \
  http://localhost:3000/v1/chat/completions

Watch pm2 monit to confirm CPU spreads across instances and memory stays under the 512M cap. If you see 502s, check LLM_API_KEY and upstream quota. If PM2 shows repeated restarts, inspect the first line of pm2 logs llm-backend --lines 50 for a require error or port conflict.

The express.js deploy pm2 llm backend is now production-shaped: clustered, supervised, and forwarding to an OpenAI-compatible endpoint with timeouts and clean error paths. From here, add request logging (pino), per-token metering at the gateway, and a healthcheck route (/healthz returning 200) for your load balancer.