Testing across multiple LLM providers usually means juggling separate API keys, base URLs, and request shapes. A postman workspace multi model llm setup collapses that complexity into one collection where you swap models via variables and reuse signed requests. This tutorial builds a working workspace from scratch using OpenAI-compatible endpoints so you can hit GPT-4o, Claude, and open-weight models without rewriting call bodies.
Prerequisites
- Postman desktop (v10+) or Newman CLI 6+ installed locally.
- One API key for an OpenAI-compatible inference gateway. A gateway such as n4n.ai collapses 240+ models behind one OpenAI-compatible endpoint, so you only store one
api_keyand change themodelvariable to route. curl(optional, for quick local verification outside Postman).- Working knowledge of bearer auth, JSON, and basic JavaScript for test scripts.
If you already have provider-specific keys, you can still follow along—just set base_url to the provider’s /v1 path and keep model names aligned with that provider.
Step 1: Create the workspace and environment
Make a new Postman workspace named LLM Multi-Model Tests. Inside it, create an environment prod-gateway with these values:
{
"name": "prod-gateway",
"values": [
{"key": "base_url", "value": "https://api.openai.com/v1", "enabled": true},
{"key": "api_key", "value": "sk-replace-me", "enabled": true},
{"key": "model", "value": "gpt-4o-mini", "enabled": true},
{"key": "temperature", "value": "0.2", "enabled": true},
{"key": "max_tokens", "value": "256", "enabled": true}
]
}If you point base_url at a multi-model gateway, keep api_key as that gateway’s token. The model string follows the gateway’s naming convention (e.g., openai/gpt-4o-mini or anthropic/claude-3-5-sonnet). Switching models later is a one-line environment edit, not a collection rewrite.
Step 2: Build the core chat request
Add a request named Core Chat with method POST and URL {{base_url}}/chat/completions. Set headers:
Authorization: Bearer {{api_key}}
Content-Type: application/jsonBody (raw JSON):
{
"model": "{{model}}",
"messages": [
{"role": "user", "content": "Explain API rate limiting in one sentence."}
],
"temperature": {{temperature}},
"max_tokens": {{max_tokens}}
}Send it. Expected 200 response:
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1710000000,
"model": "gpt-4o-mini",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Rate limiting restricts request frequency per key to prevent overload and ensure fair use."
},
"finish_reason": "stop"
}
],
"usage": {"prompt_tokens": 12, "completion_tokens": 18, "total_tokens": 30}
}That single request is the backbone of your postman workspace multi model llm testing—every other scenario reuses it with different variables or small body tweaks.
Step 3: Drive multiple models with a data file
Create models.csv in your repo:
model
gpt-4o-mini
claude-3-5-sonnet
llama-3.1-70b-instruct
mistral-large-latestIn Collection Runner, select the prod-gateway environment, load models.csv as data, and run Core Chat. Postman overrides {{model}} per iteration. Invalid model names return 4xx and the run continues, which is exactly what you want for broad coverage.
For headless execution:
newman run llm-multi-model.postman_collection.json -e env.prod-gateway.json -d models.csvSample per-iteration output:
→ Core Chat
POST https://api.openai.com/v1/chat/completions [200 OK, 412ms]
✓ status is 200
✓ has completion text
→ Core Chat
POST https://api.openai.com/v1/chat/completions [200 OK, 689ms]
✓ status is 200
✓ has completion textStep 4: Add response contract tests
Open the Tests tab of Core Chat. Paste:
pm.test("status is 200", () => pm.response.to.have.status(200));
pm.test("returns non-empty content", () => {
const body = pm.response.json();
pm.expect(body.choices).to.be.an('array').and.not.empty;
pm.expect(body.choices[0].message.content).to.be.a('string').and.not.empty;
});
pm.test("usage is metered", () => {
const body = pm.response.json();
pm.expect(body.usage.total_tokens).to.be.a('number').and.above(0);
});
const latency = pm.response.responseTime;
pm.environment.set("last_latency", latency);
console.log(`Model ${pm.variables.get("model")} latency: ${latency}ms`);Run the collection again. Console shows:
Model gpt-4o-mini latency: 412ms
Model claude-3-5-sonnet latency: 689ms
Model llama-3.1-70b-instruct latency: 521msThese assertions catch provider schema drift early—a real risk when a postman workspace multi model llm suite talks to heterogeneous backends that each implement the OpenAI shape with slight deviations.
Step 5: Isolate scenarios with folders
Create a folder JSON Mode. Duplicate Core Chat, rename to JSON forced, and change the body:
{
"model": "{{model}}",
"messages": [
{"role": "user", "content": "Return a JSON object with key 'status' and value 'ok'."}
],
"response_format": {"type": "json_object"},
"temperature": 0
}Add a test:
pm.test("body parses as JSON with expected key", () => {
const txt = pm.response.json().choices[0].message.content;
let parsed;
pm.expect(() => { parsed = JSON.parse(txt); }).to.not.throw();
pm.expect(parsed.status).to.equal("ok");
});Add a Streaming folder if your gateway supports SSE. Set "stream": true and verify the first delta chunk carries a role or content field. Streaming across models is where you’ll find subtle differences in delta formats and finish reasons.
Step 6: Inject routing and cache headers
Some gateways accept client routing directives. n4n.ai honors client routing directives and forwards provider cache-control hints, so you can pin a provider or reuse prompt caches directly from Postman headers. In the Core Chat Headers tab add:
x-router-prefer: openai
cache-control: max-age=600If the gateway supports it, x-router-prefer biases model selection toward a specific upstream; cache-control tells the provider to honor prompt caching where available. Remove these for default round-robin or fallback behavior.
Automatic fallback is useful when a provider is rate-limited: the gateway returns a successful completion from a secondary provider if configured. Your Postman tests should assert on body.model to record which backend actually served the token, because the requested model string may differ from the fulfilled one.
Step 7: Export and version control
Postman collections are portable JSON. Commit them alongside the data file:
git add llm-multi-model.postman_collection.json env.prod-gateway.json models.csv
git commit -m "add postman workspace multi model llm test suite"Run in CI with Newman and fail the pipeline on assertion errors:
newman run llm-multi-model.postman_collection.json -e env.prod-gateway.json -d models.csv --bailA non-zero exit blocks merges when a model regresses or a provider changes its response shape.
Step 8: Dynamic model discovery via pre-request
OpenAI-compatible gateways expose GET /models. Use a pre-request script on the collection to pull the live model list and store it:
pm.sendRequest({
url: pm.environment.get("base_url") + "/models",
headers: {Authorization: "Bearer " + pm.environment.get("api_key")}
}, (err, res) => {
if (!err && res.code === 200) {
const ids = res.json().data.map(m => m.id);
pm.environment.set("available_models", JSON.stringify(ids));
console.log("Discovered " + ids.length + " models");
} else {
console.warn("Model list fetch failed: " + (err || res.code));
}
});Expected console line:
Discovered 240 modelsYou can later feed that array into a data file generator or just inspect it to keep models.csv current. Gateways with per-token usage metering return usage objects you can assert on to track cost across models without leaving Postman.
What you have now
You built a parameterized Postman collection that exercises several LLMs through one OpenAI-compatible shape. Environment variables swap credentials and base URLs; a CSV drives model iteration; tests enforce response contracts and record latency. With minimal additions you can cover function calling, vision inputs, and token metering. The postman workspace multi model llm pattern keeps your integration tests honest as providers change underneath you.