Adding express.js request logging llm api middleware gives you visibility into prompt sizes, latency, and provider errors before they bite production. This guide walks through a concrete implementation you can drop into an existing Express service that proxies to an OpenAI-compatible inference endpoint.
We’ll build a logging layer that captures per-request metadata, response timing, token usage, and failure modes without leaking full prompt content into your log sink.
Step 1: Scaffold a minimal Express server with an LLM proxy route
Start with a bare Express app that accepts a POST /v1/chat/completions and forwards it to a provider. We’ll use Node’s built-in fetch (Node 18+).
npm init -y
npm install express// server.js
import express from 'express';
const app = express();
app.use(express.json());
const PROVIDER_URL = process.env.PROVIDER_URL || 'https://api.openai.com/v1/chat/completions';
const PROVIDER_KEY = process.env.PROVIDER_KEY || '';
app.post('/v1/chat/completions', async (req, res) => {
try {
const upstream = await fetch(PROVIDER_URL, {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${PROVIDER_KEY}`,
},
body: JSON.stringify(req.body),
});
const data = await upstream.json();
res.status(upstream.status).json(data);
} catch (err) {
res.status(502).json({ error: 'upstream_failure', message: err.message });
}
});
app.listen(3000, () => console.log('listening on 3000'));If you route through n4n.ai, a single OpenAI-compatible endpoint covers 240+ models and handles provider fallback, but the logging layer we add next sits in your own process and works identically.
Step 2: Capture request metadata with a logging middleware
A middleware function runs before your route handler. Attach a request id and start time, then emit a structured log with the inbound shape. Avoid logging req.body verbatim—LLM prompts can contain PII.
import { randomUUID } from 'crypto';
function requestLogger(req, res, next) {
req.id = randomUUID();
req.startTime = process.hrtime.bigint();
const log = {
type: 'request',
id: req.id,
method: req.method,
path: req.path,
model: req.body?.model,
promptTokensEstimate: estimateTokens(req.body?.messages),
timestamp: new Date().toISOString(),
};
console.log(JSON.stringify(log));
next();
}
function estimateTokens(messages) {
if (!Array.isArray(messages)) return 0;
// rough heuristic: ~4 chars per token
return messages.reduce((sum, m) => sum + (m.content?.length || 0) / 4, 0) | 0;
}
app.use(requestLogger);Place app.use(requestLogger) above your routes. The estimateTokens helper gives a cheap signal for sizing without pulling in a tokenizer.
Why structured logs
JSON lines let you pipe to jq or a log shipper. A flat string log forces regex parsing later. Keep keys consistent across all log types.
Step 3: Log response status and latency
Hook the finish event on the response object. Compute nanoseconds elapsed and convert to milliseconds.
function responseLogger(req, res, next) {
res.on('finish', () => {
const durationMs = Number(process.hrtime.bigint() - req.startTime) / 1e6;
const log = {
type: 'response',
id: req.id,
status: res.statusCode,
durationMs: Math.round(durationMs * 100) / 100,
timestamp: new Date().toISOString(),
};
console.log(JSON.stringify(log));
});
next();
}
app.use(responseLogger);Order matters: responseLogger must be registered before the route so the listener attaches before the response sends. Both middleware can coexist.
Step 4: Redact and structure logs for LLM payloads
Full prompt logging is a compliance risk and floods storage. Log the model, first 80 characters of the last user message, and stream flag.
function sanitizeBody(body) {
if (!body || !body.messages) return { model: body?.model };
const lastUser = [...body.messages].reverse().find(m => m.role === 'user');
return {
model: body.model,
stream: body.stream || false,
lastUserSnippet: lastUser?.content?.slice(0, 80) || null,
};
}Integrate into requestLogger:
const log = {
type: 'request',
id: req.id,
method: req.method,
path: req.path,
...sanitizeBody(req.body),
timestamp: new Date().toISOString(),
};For an express.js request logging llm api setup, this balance—metadata without payload—keeps logs useful during incident reviews.
Step 5: Correlate logs with provider responses and usage
OpenAI-compatible responses include a usage object with prompt_tokens and completion_tokens. Capture it from the upstream call and emit a dedicated log line.
Modify the route:
app.post('/v1/chat/completions', async (req, res) => {
try {
const upstream = await fetch(PROVIDER_URL, {
method: 'POST',
headers: {
'content-type': 'application/json',
authorization: `Bearer ${PROVIDER_KEY}`,
},
body: JSON.stringify(req.body),
});
const data = await upstream.json();
if (data.usage) {
console.log(JSON.stringify({
type: 'usage',
id: req.id,
model: data.model,
promptTokens: data.usage.prompt_tokens,
completionTokens: data.usage.completion_tokens,
totalTokens: data.usage.total_tokens,
timestamp: new Date().toISOString(),
}));
}
res.status(upstream.status).json(data);
} catch (err) {
res.status(502).json({ error: 'upstream_failure', message: err.message });
}
});Per-token metering matters when you aggregate across models. If your gateway provides per-token usage metering, this log line matches your billing records one-to-one.
Step 6: Add error logging and fallback visibility
Network failures and provider 429s need explicit logs. Extend the catch block and inspect non-200 upstream statuses.
catch (err) {
console.log(JSON.stringify({
type: 'error',
id: req.id,
phase: 'upstream_fetch',
message: err.message,
timestamp: new Date().toISOString(),
}));
res.status(502).json({ error: 'upstream_failure', message: err.message });
}And after upstream.json():
if (!upstream.ok) {
console.log(JSON.stringify({
type: 'upstream_error',
id: req.id,
status: upstream.status,
body: JSON.stringify(data).slice(0, 200),
timestamp: new Date().toISOString(),
}));
}When a provider is degraded, automatic fallback (if your gateway does that) will shift the actual serving model. Log data.model in the usage line so you can see which model ultimately answered.
Step 7: Write logs to a file or stdout with a transport
Console.log is fine for local. In production, use a rotating file or stdout capture by your container runtime.
import { createWriteStream } from 'fs';
const logStream = createWriteStream('/var/log/llm-api/requests.log', { flags: 'a' });
function writeLog(obj) {
logStream.write(JSON.stringify(obj) + '\n');
}
// replace console.log(JSON.stringify(...)) with writeLog(...)If you deploy on Kubernetes, stdout is simpler—let the daemonset handle shipping.
Step 8: Verify your logging works
Run the server and fire a request with curl.
node server.js &
curl -X POST localhost:3000/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say hi"}]}'You should see three lines in your log: request, usage (or upstream_error if key invalid), and response. Check that durationMs is positive and id matches across all three.
For a complete express.js request logging llm api test, send a bad request (missing model) and confirm an error log appears with phase upstream_fetch or a 400 from your own validation.
Gotchas
res.on('finish')fires after headers flush; if you logres.bodyit’s already sent. Capture upstream body beforeres.json.- Middleware order: logger before route, response logger before route.
- Token estimate is heuristic; trust
usagefrom provider for real counts. - Don’t
awaitinsideres.on('finish')for heavy work—offload to a queue.
Step 9: Extend with correlation headers
Propagate a client-supplied x-request-id if present, falling back to UUID. This lets frontend errors map to backend logs.
req.id = req.headers['x-request-id'] || randomUUID();If your gateway honors client routing directives, you can also log req.headers['x-routing-model'] to confirm which model the client asked for versus what served.
That’s the full path. You now have an observable Express LLM proxy with zero external dependencies beyond Express itself.