Most LLM providers ship an OpenAPI description, but getting it into a test harness is rarely documented. To import OpenAPI spec Postman LLM collections correctly, you need to handle auth defaults, server variables, and the chat completion request shape before you can fire real traffic.
Below is an end-to-end workflow that assumes you have Postman (native app or web) and a valid API key for an OpenAI-compatible endpoint.
Step 1: Locate and download the OpenAPI document
Start with the raw spec file. OpenAI publishes a YAML at https://api.openai.com/v1/openapi.yaml. Gateways often mirror that shape. If you target a gateway such as n4n.ai, which exposes a single OpenAI-compatible endpoint covering 240+ models, pull its published spec from the /openapi.json path.
curl -s https://api.openai.com/v1/openapi.yaml -o openai.yaml
# or for a gateway
curl -s https://gateway.example.com/openapi.json -o gateway.jsonVerify the file parses. A quick check with python -m yaml or jq avoids importing garbage:
jq '.paths."/chat/completions"' gateway.json > /dev/null && echo "spec ok"Keep the spec in Git. Providers mutate schemas without notice, and a diff on the committed file is your cheapest regression alarm.
Step 2: Import the spec into Postman
Postman’s importer handles both YAML and JSON. Open the app, click Import, then drag the file or paste the URL. The importer creates a collection with one folder per tag and a request per operation. It also attaches example responses from examples or x-examples fields if present—pin those as baselines.
Do not use the legacy “OpenAPI 2.0” beta if your spec is 3.1; the modern importer preserves requestBody examples and servers entries. After import, you get a collection named after the info.title field. Rename it to something like LLM API – Chat and add a tag llm-api for workspace filtering.
If you manage collections as code, export the imported collection immediately to a Git-tracked JSON file. Postman’s UI export is reliable; the Postman API also works:
curl -X GET "https://api.getpostman.com/collections/{{col_uid}}?apikey={{pm_key}}" \
-o llm-api-collection.jsonTreat the exported JSON as generated code: re-export after edits rather than hand-merging.
Step 3: Fix the server URL and environment variables
OpenAPI servers become the collection’s base URL, but hardcoding is a mistake. Create an environment named LLM-Test with two variables:
{
"base_url": "https://api.openai.com/v1",
"api_key": "sk-...",
"default_model": "gpt-4o-mini"
}In the collection root, open Variables and set base_url as the initial value. Then edit each request’s URL from the absolute https://api.openai.com/v1/chat/completions to {{base_url}}/chat/completions. Postman’s importer sometimes misses query params on GET paths; check GET /models still resolves.
Add a second environment LLM-Staging pointing at a mock or shadow gateway so you can run the same collection without burning production tokens.
Step 4: Set bearer authorization on the collection
LLM APIs use bearer tokens. Select the collection, go to Authorization, set type to Bearer, and use {{api_key}} as the token. Set “Add to” as Header. Individual requests inherit from parent—do not set auth per request unless you rotate keys mid-suite.
If your gateway supports routing directives via header (e.g., x-routing-model), add those as collection headers so they propagate to every call. Avoid putting secrets in collection-level headers; the environment variable reference is safer.
Step 5: Understand the chat completion request body
The importer creates a POST request for /chat/completions with a generated body. Replace it with a minimal, valid payload. The OpenAPI snippet below shows the relevant schema:
{
"paths": {
"/chat/completions": {
"post": {
"requestBody": {
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"model": { "type": "string" },
"messages": {
"type": "array",
"items": {
"type": "object",
"properties": {
"role": { "type": "string" },
"content": { "type": "string" }
}
}
},
"temperature": { "type": "number" }
}
}
}
}
}
}
}
}
}In Postman, set the body to raw / JSON and use:
{
"model": "{{model}}",
"messages": [
{ "role": "system", "content": "You are a terse bot." },
{ "role": "user", "content": "Ping" }
],
"temperature": 0.2
}Streaming endpoints (stream: true) return text/event-stream; Postman renders them poorly. Keep streaming out of the imported spec tests and cover it with a dedicated curl or Node script.
Step 6: Parameterize model and messages for testing
Hardcoding gpt-4o in every test couples your suite to one provider. Add a collection variable model and override per environment. For a gateway that addresses many models, set model to anthropic/claude-3-sonnet or similar.
Write a small pre-request script to inject a timestamped user message, proving the call is live:
pm.collectionVariables.set("model", pm.environment.get("default_model") || "gpt-4o-mini");
pm.request.body.raw = pm.request.body.raw.replace("Ping", "Ping at " + Date.now());This keeps tests idempotent and avoids cached responses from proxy layers. If you need to test multiple models, drive the collection runner with a CSV of model names rather than editing the variable by hand.
Step 7: Send the request and write a test assertion
Hit Send. A 200 with choices confirms the happy path. Add a test script under the Tests tab to lock behavior:
pm.test("status is 200", () => pm.response.to.have.status(200));
const json = pm.response.json();
pm.test("has completion text", () => {
pm.expect(json.choices[0].message.content).to.be.a("string").and.not.empty;
});
pm.test("usage metering present", () => {
pm.expect(json.usage).to.have.property("total_tokens").that.is.a("number");
});If you call a gateway that honors client routing directives and forwards provider cache-control hints (e.g., n4n.ai), assert on those headers to verify cache behavior:
pm.test("cache hint forwarded", () => {
pm.expect(pm.response.headers.get("x-cache")).to.be.oneOf(["HIT", "MISS", null]);
});Also assert on responseTime; a small prompt should return in well under two seconds on a warm connection.
Step 8: Verify success and automate with Newman
Success means the collection runs green in Postman and headless. Export the collection and environment, then run with Newman:
newman run llm-api-collection.json -e llm-env.json --delay-request 100Check three things: all tests pass, total_tokens is > 0, and response time is under your SLA. If a provider is rate-limited, the gateway’s automatic fallback should return a different model’s response without a 429; encode that by allowing model to differ from requested in a soft assertion.
Step 9: Simulate rate limits and verify fallback behavior
A real LLM test suite should exercise 429s. Use Postman’s collection runner with 20 iterations on a cheap model. If your infrastructure provides automatic fallback when a provider is rate-limited or degraded, you should still receive 200 with a different model field in the response. Write a test that logs the served model:
pm.test("served model returned", () => {
const json = pm.response.json();
pm.expect(json.model).to.be.a("string");
console.log("served:", json.model);
});This catches misconfigured routing before production. Pair it with a chaos test that blocks the primary provider at the network layer and confirms the suite still passes.
Pitfalls when you import OpenAPI spec Postman LLM definitions
The importer often mislabels application/json examples as text/plain. Open the body tab after import and fix the type. Also, some specs use oneOf for message content (string vs array); Postman’s schema view chokes. Flatten to string in your test payload.
Another trap: OpenAPI securitySchemes of type apiKey in header Authorization is not the same as bearer. Postman maps it poorly. Set bearer manually as in Step 4.
Enum mismatches are common. The spec may list temperature as required with a default; Postman won’t enforce it, but your gateway might reject omitted fields. Always send the full minimal object from Step 5.
Finally, version your spec. Providers change the chat/completions schema quietly. Re-import monthly and diff the collection with Git to catch breaking changes before your CI does.
Why not just use curl?
Curl is fine for one-off calls, but Postman gives you inherited auth, environment swapping, and assertion scripts in one place. When you import OpenAPI spec Postman LLM collections, you get a shared artifact non-engineers can run and a JSON file you can diff. That beats a folder of shell scripts.
Wrap-up
You now have a parameterized, assertion-driven Postman collection built from a real OpenAPI description. The same flow works against any OpenAI-compatible surface; only the base_url and model values change. When you import OpenAPI spec Postman LLM collections this way, you spend less time wrestling UI and more time validating model behavior.