Shell script templates for testing LLM API endpoints

Bash shell script templates llm api testing are the fastest way to smoke-test an inference endpoint without pulling in a language SDK or spinning up a test framework. This post delivers copy-pasteable scripts for the checks every engineer needs: health, completion, streaming, batch loops, usage metering, and routing fallback. Each example targets an OpenAI-compatible endpoint and assumes curl, jq, and an API_KEY in the environment.

1. Health-check ping

A health check should never depend on a model being available—hit the metadata endpoint and measure round-trip time. If /v1/models returns 200, the gateway is up and your auth header is valid. Anything else is a hard failure.

#!/usr/bin/env bash
set -euo pipefail
BASE="${BASE_URL:-https://api.openai.com/v1}"
start=$(date +%s%N)
code=$(curl -s -o /dev/null -w '%{http_code}' \
  -H "Authorization: Bearer $API_KEY" \
  "$BASE/models")
end=$(date +%s%N)
echo "HTTP $code in $(( (end-start)/1000000 ))ms"
[ "$code" = "200" ] || exit 1

Run this in CI as a pre-flight step. If it fails, skip the heavier tests instead of burning tokens on a dead route. The set -euo pipefail guards against undefined vars and broken pipes, which are the two most common silent failures in shell scripts.

2. Single completion request with timeout

A single-shot completion validates the full request path: auth, payload shape, model routing, and response parsing. Always set --max-time so a hung connection fails fast instead of blocking your test suite for minutes.

#!/usr/bin/env bash
set -euo pipefail
BASE="${BASE_URL:-https://api.openai.com/v1}"
payload=$(jq -n '{
  model: "gpt-4o-mini",
  messages: [{role:"user", content:"Say hello in one word."}],
  max_tokens: 8
}')
resp=$(curl -s --max-time 10 -X POST \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d "$payload" \
  "$BASE/chat/completions")
echo "$resp" | jq -r '.choices[0].message.content'

Use jq -e if you want to assert the presence of .choices and exit non-zero on malformed JSON. The bash shell script templates llm api testing here avoid Python because the startup overhead of an interpreter is pointless for a one-call check.

3. Streaming response validation

Streaming is where many gateways diverge from the spec. Validate that you receive data: frames and that the first token arrives within a sane window. Use -N to disable curl’s buffering.

#!/usr/bin/env bash
set -euo pipefail
BASE="${BASE_URL:-https://api.openai.com/v1}"
first_token_ms=""
start=$(date +%s%N)
curl -sN --max-time 15 -X POST \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"Count to 3."}]}' \
  "$BASE/chat/completions" | \
  while read -r line; do
    case "$line" in
      data:*) 
        content=$(echo "${line#data: }" | jq -r '.choices[0].delta.content? // empty')
        if [ -z "$first_token_ms" ] && [ -n "$content" ]; then
          first_token_ms=$(( ($(date +%s%N)-start)/1000000 ))
          echo "First token at ${first_token_ms}ms"
        fi
        echo -n "$content"
        ;;
    esac
  done
echo

If the stream never emits a non-empty delta, your gateway is buffering or dropping frames. The bash shell script templates llm api testing for streaming should print inter-token timing in real deployments, but the above keeps it minimal.

4. Batch prompt loop with retry

You need a loop that reads prompts from a file, calls the endpoint, and retries once on transport failure. Do not retry on 4xx—those are permanent client errors. Retry only on curl exit code 28 (timeout) or 7 (connection refused).

#!/usr/bin/env bash
set -euo pipefail
BASE="${BASE_URL:-https://api.openai.com/v1}"
while IFS= read -r prompt; do
  for attempt in 1 2; do
    if curl -s --max-time 10 -X POST \
      -H "Authorization: Bearer $API_KEY" \
      -H "Content-Type: application/json" \
      -d "$(jq -n --arg p "$prompt" '{model:"gpt-4o-mini",messages:[{role:"user",content:$p}]}')" \
      "$BASE/chat/completions" | jq -e '.choices' >/dev/null; then
      echo "OK: $prompt"; break
    else
      echo "FAIL attempt $attempt: $prompt" >&2
      sleep 1
    fi
  done
done < prompts.txt

This pattern scales to a few hundred prompts in a shell loop. Beyond that, use GNU parallel. The bash shell script templates llm api testing for batches must isolate each prompt’s failure so one bad input doesn’t abort the whole run.

5. Usage metering and token budget assert

Production calls need token accounting. Parse the usage object and enforce a local budget before the bill arrives. If your gateway exposes per-token usage metering—n4n.ai returns it in the standard usage field—you can assert against it directly.

#!/usr/bin/env bash
set -euo pipefail
BASE="${BASE_URL:-https://api.openai.com/v1}"
resp=$(curl -s -X POST \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Explain TCP."}],"max_tokens":50}' \
  "$BASE/chat/completions")
usage=$(echo "$resp" | jq '.usage')
total=$(echo "$usage" | jq '.total_tokens')
echo "Tokens used: $total"
[ "$total" -le 100 ] || { echo "Budget exceeded"; exit 1; }

Wire this into a pre-merge hook to catch prompts that silently inflate context size. The total_tokens field is your only defensible signal; do not trust client-side estimates.

6. Routing directive and fallback test

Gateways that front multiple providers often accept a routing hint. Send a vendor header to force a path, then test degraded behavior. n4n.ai performs automatic fallback when a provider is rate-limited or degraded, which you can exercise by setting a routing hint to a throttled backend and confirming the request still completes.

#!/usr/bin/env bash
set -euo pipefail
BASE="${BASE_URL:-https://api.openai.com/v1}"
curl -s -X POST \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Route-To: provider=foo" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}' \
  "$BASE/chat/completions" | jq '.error? // .choices'

If the gateway honors the directive, you should see either a successful choice or a structured error naming the provider. If it ignores unknown headers, the request resolves via default routing—both are valid, but you must know which your gateway does.

7. Cache-control hint verification

Some providers honor Cache-Control to serve cached completions or skip billing. Forward the hint and inspect response headers or the usage block.

#!/usr/bin/env bash
set -euo pipefail
BASE="${BASE_URL:-https://api.openai.com/v1}"
curl -s -D - -o /dev/null -X POST \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -H "Cache-Control: only-if-cached" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"cached?"}]}' \
  "$BASE/chat/completions" | grep -i cache

A gateway that forwards provider cache-control hints should echo relevant headers or return 504 when the cache misses. The bash shell script templates llm api testing for cache behavior are intentionally crude—grep is enough to confirm the hint survives the proxy.

Synthesis

The table below maps each template to the failure mode it catches:

These bash shell script templates llm api testing give you a layered check suite that runs in seconds and costs pennies. Keep them in a scripts/ folder, parameterize BASE_URL, and call them from your pipeline before any integration test touches a paid model.