How to test LLM APIs with Insomnia

Most engineers reach for curl or a quick Python script, but if you test LLM API Insomnia collections, you get persisted requests, environment switching, and response diffing without leaving a GUI. Insomnia treats OpenAI-compatible endpoints as ordinary REST calls; the only wrinkles are streaming MIME types and JSON schema expectations. This guide walks through a complete setup from blank workspace to CI-ready automated checks.

Step 1: Install Insomnia and create a workspace

Download Insomnia Desktop (or just the inso CLI for headless work). Create a new “Document” — the current Insomnia term for a collection — and name it llm-tests. Inside, create a folder chat to group related requests. The document is stored as versionable JSON/YAML on disk, so you can commit it to git like any other source file.

Do not skip the folder step. When you accumulate 30 requests against different models and endpoints, a flat list becomes unmaintainable.

Verify success: The left sidebar shows llm-tests with an empty chat folder. No requests exist yet, and the document saves without error.

Step 2: Define environment variables

LLM providers share the same request shape but differ in base URL and auth header. Insomnia environments let you swap these without editing each request. Open “Manage Environments” and create Local and Production.

Use this structure for Production:

{
  "base_url": "https://api.openai.com/v1",
  "api_key": "sk-your-real-key",
  "model": "gpt-4o-mini"
}

Mark api_key as “Secret” so it is masked in the UI and excluded from plain exports. When you test LLM API Insomnia setups across a team, store the key in a vault and inject it at runtime via inso instead of committing it.

Create a Local environment pointing at a mock or reverse proxy if you have one. The variable names stay identical; only values change.

Verify success: The environment dropdown shows Production active. Hovering {{ base_url }} in any request field resolves to the full URL.

Step 3: Send a non-streaming chat completion

Inside chat, create a POST request named Basic Chat. Set the URL to {{ base_url }}/chat/completions. Add headers:

Authorization: Bearer {{ api_key }}
Content-Type: application/json

Paste this body:

{
  "model": "{{ model }}",
  "messages": [
    {"role": "user", "content": "Return a JSON object with a single key 'ok' and boolean true."}
  ],
  "temperature": 0.1,
  "max_tokens": 50
}

Send. You should receive a 200 with choices[0].message.content containing parseable JSON. Note the usage object at the bottom — that is your per-token metering signal and should be present on every non-streaming call.

Verify success: Status code 200 OK. The response preview shows {"ok": true} or similar inside the message content. Save the request. If you see 401, the env is wrong; 422 means a missing required field.

Step 4: Test streaming responses

Streaming is where hand-rolled scripts break. In Insomnia, flip stream to true in the same body:

{
  "model": "{{ model }}",
  "messages": [{"role": "user", "content": "Count to 5 slowly."}],
  "stream": true
}

Insomnia renders text/event-stream as sequential data: {...} frames. Each frame carries choices[0].delta. The usage field, if the provider supports it, arrives in a final event before data: [DONE].

Use the built-in “Filter” box with $.choices[0].delta.content to extract just the text fragments during debugging.

Verify success: The response pane shows multiple data: lines, each with a delta. The last line is data: [DONE]. If you see only one frame, the provider ignored stream or a proxy buffered the response.

Step 5: Test multiple models behind one gateway

Switching base URLs per provider wastes engineering time. If you route through n4n.ai, one OpenAI-compatible endpoint addresses 240+ models and automatically falls back when a provider is rate-limited or degraded. It honors client routing directives and forwards provider cache-control hints, so you can test cache behavior without custom middleware.

Create a Gateway environment:

{
  "base_url": "https://api.n4n.ai/v1",
  "api_key": "sk-gateway-key",
  "model": "anthropic/claude-3.5-sonnet"
}

The same Basic Chat request now hits a different backend by changing only model. To force a specific provider path, pass provider/model; the gateway forwards appropriately. If your provider supports prompt caching, send the relevant header (e.g., cache-control: max-age=300 where supported) and it passes through unchanged.

Verify success: Switch to Gateway, send Basic Chat with {{ model }} set to openai/gpt-4o. A 200 returns with the expected shape. Disable one provider key to observe fallback — the response still succeeds, often with a different model field echoed. When you test LLM API Insomnia workflows at scale, this eliminates per-vendor collections.

Step 6: Add response assertions

Manual eyeballing fails in CI. Insomnia supports assertions via the “Assertions” tab on a response. For Basic Chat, add:

• Status equals 200
• Body path choices is an array with length > 0
• Body path choices[0].message.content is a string

In the GUI, click “Assertions” → “New Assertion” and pick from the dropdown. For scripted control, export the document and run inso:

inso run test "llm-tests" --env "Production"

If you need custom logic (e.g., validate that content is JSON), use the request’s “Tests” field:

const body = JSON.parse(response.body);
if (!body.choices || !body.choices[0].message.content) {
  throw new Error("Missing content");
}

Verify success: The assertion list shows green checks. inso run test exits 0.

Step 7: Automate with the Insomnia CLI

Commit the Insomnia document to git. In CI, install inso, then run requests or tests headlessly:

npm i -g insomnia-inso

# run a single request
inso run request "Basic Chat" --env "Gateway"

# run full test suite
inso run test "llm-tests" --env "Production"

Set api_key from a CI secret so it never hits disk. For parallel smoke tests, loop over models:

for m in gpt-4o-mini anthropic/claude-3.5-sonnet meta/llama-3.1-70b; do
  inso run request "Basic Chat" --env "Gateway" --var model=$m
done

Verify success: CI log shows 200 for each model and the test suite passes. If a provider is degraded, gateway fallback keeps the run green — exactly what you want from a test llm api insomnia pipeline.

Step 8: Test structured outputs and JSON mode

Production LLM integrations rarely want free text. OpenAI-compatible APIs accept response_format to force JSON. Update the body:

{
  "model": "{{ model }}",
  "messages": [{"role": "user", "content": "Give me a user object."}],
  "response_format": {"type": "json_object"},
  "temperature": 0
}

Some gateways pass this through to any backend that supports it; others reject it for models that don’t. Treat a 400 with invalid_response_format as a signal to branch your test by model capability.

Verify success: Response content is strict JSON, no markdown fences. Add an assertion that JSON.parse(choices[0].message.content) does not throw.

Troubleshooting

• 401 Unauthorized: Key missing or env not selected. Hover {{ api_key }} to confirm resolution.
• 404 Model not found: Model string invalid for that base URL. Gateway accepts provider/model; bare OpenAI does not.
• Stream hangs: Corporate proxies often buffer SSE. Test locally first.
• 422: Missing messages or max_tokens too low. Inspect the error detail field.
• 400 on response_format: Backend lacks JSON mode. Skip that assertion for unsupported models.

Treat Insomnia as an executable spec, not just a GUI. Once the collection lives in git and inso runs in CI, you have a regression shield for prompt and model changes — and the next time you test LLM API Insomnia requests, the work is already done.