n4nAI
Open console →
API integrationHow-tocURL LLM API Cookbook

Streaming chat completions from the command line with curl

Learn how to run curl streaming chat completions command line against an OpenAI-compatible API, parse SSE, and verify token streaming end to end.

n4n Team4 min read812 words

Audio narration

Coming soon — every post will get a voice note here.

Most LLM APIs document streaming for SDKs but leave the raw protocol vague. If you need to debug a gateway or script a quick pipeline, curl streaming chat completions command line is the fastest way to see exactly what the server sends over the wire.

Step 1: Export credentials and choose an OpenAI-compatible base URL

Set the key and endpoint in your shell so you are not pasting secrets into every command. Any server that implements the OpenAI chat completions shape works; the examples below target the standard /v1/chat/completions path.

export API_KEY="sk-..."
export BASE_URL="https://api.openai.com/v1"
# Or point at a gateway:
# export BASE_URL="https://api.n4n.ai/v1"

If you are using n4n.ai, the same OpenAI-compatible endpoint fronts 240+ models and will automatically fall back when a provider is rate-limited or degraded, but the curl mechanics are identical.

Step 2: Validate auth with a non-streaming call

Before dealing with chunked responses, confirm your key and model name are correct. Send a minimal JSON body with stream omitted (defaults to false).

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Say hi in 3 words."}]
  }'

You should get a single JSON object with a choices[0].message.content field. If this fails with 401, your key is wrong; 404 means the model or path is off. Fix these before touching streams—debugging auth inside an SSE flow is annoying.

Step 3: Enable streaming and disable curl buffering

Streaming uses Server-Sent Events (SSE). The server sends data: {json}\n\n lines until data: [DONE]. Curl will buffer by default; pass -N (or --no-buffer) to flush immediately. HTTP/2 is fine, but if you see odd interleaving, force HTTP/1.1 with --http1.1.

curl -N "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "stream": true,
    "messages": [{"role": "user", "content": "Count to 5 slowly."}]
  }'

Raw output looks like:

data: {"id":"chatcmpl-...","choices":[{"delta":{"role":"assistant"},"index":0}]}
data: {"id":"chatcmpl-...","choices":[{"delta":{"content":"1"},"index":0}]}
data: {"id":"chatcmpl-...","choices":[{"delta":{"content":" "},"index":0}]}
data: {"id":"chatcmpl-...","choices":[{"delta":{"content":"2"},"index":0}]}
...
data: [DONE]

Each delta carries only the new token(s). The first event often contains role; subsequent ones contain content. Some providers also send finish_reason on the last data line before [DONE]. Ignore it for text extraction, but log it if you care about truncation.

Step 4: Extract just the text with a parser

Grepping for content is fragile because JSON spans a single line per event but includes nested braces. Two practical options:

Option A: sed + jq

Strip the data: prefix and let jq pick the field:

curl -N "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"Say a haiku."}]}' \
  | sed 's/^data: //' \
  | jq -r 'select(.choices[0].delta.content != null) | .choices[0].delta.content' \
  | tr -d '\n'

This works but breaks if the server sends a comment line (:: ping) or whitespace keep-alives.

Option B: Small Python script (robust)

A 20-line reader handles keep-alives and UTF-8 splits safely:

import sys, json

for line in sys.stdin:
    line = line.strip()
    if not line.startswith("data:"):
        continue
    payload = line[len("data:"):].strip()
    if payload == "[DONE]":
        break
    if not payload or payload.startswith(":"):
        continue
    try:
        obj = json.loads(payload)
    except json.JSONDecodeError:
        continue
    delta = obj["choices"][0].delta
    if "content" in delta:
        sys.stdout.write(delta["content"])
        sys.stdout.flush()
print()

Pipe curl into it:

curl -N "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"Say a haiku."}]}' \
  | python3 extract.py

You now have a clean terminal printout of the model’s output as it arrives.

Step 5: Send routing and cache hints

Gateways often accept headers to pin a provider or control caching. n4n.ai honors client routing directives and forwards provider cache-control hints, so you can test cache behavior with a standard header:

curl -N "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -H "Cache-Control: max-age=600" \
  -d '{
    "model": "anthropic/claude-3.5-sonnet",
    "stream": true,
    "messages": [{"role":"user","content":"Repeat: cached response test."}]
  }'

If the upstream provider supports prompt caching, the gateway passes the hint through; you will not see it in the SSE stream, but subsequent identical requests should return faster and the final usage chunk may report cache read tokens. Vendor-specific routing headers vary—check your gateway docs before assuming a header name.

Step 6: Capture usage and verify success

Even with streaming, many servers send a final event containing usage. Look for it in the raw stream:

curl -N "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","stream":true,"messages":[{"role":"user","content":"What is 2+2?"}]}' \
  | grep -o '"usage":{[^}]*}'

A successful run shows:

  • Multiple data: lines with delta.content appearing incrementally.
  • A terminating data: [DONE].
  • A usage object with prompt_tokens, completion_tokens, and total_tokens.

Some OpenAI-compatible servers omit usage unless you explicitly request it:

{
  "model": "gpt-4o-mini",
  "stream": true,
  "stream_options": {"include_usage": true},
  "messages": [{"role":"user","content":"Hi"}]
}

If you are on a metered gateway, per-token usage metering means those numbers are what you will be billed on, so confirm they appear before trusting a pipeline. Extend the Python parser to print usage at the end:

import sys, json

usage = None
for line in sys.stdin:
    line = line.strip()
    if not line.startswith("data:"):
        continue
    payload = line[len("data:"):].strip()
    if payload == "[DONE]":
        break
    if not payload or payload.startswith(":"):
        continue
    obj = json.loads(payload)
    if "usage" in obj:
        usage = obj["usage"]
    delta = obj["choices"][0].delta
    if "content" in delta:
        sys.stdout.write(delta["content"])
        sys.stdout.flush()
print()
if usage:
    print("\nUSAGE:", json.dumps(usage), file=sys.stderr)

Step 7: Wrap it in a reusable shell function

Put this in your .bashrc to avoid retyping flags:

llm-stream() {
  local prompt="$1"
  local model="${2:-gpt-4o-mini}"
  curl -N "$BASE_URL/chat/completions" \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"model\":\"$model\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"$prompt\"}]}" \
    | python3 extract.py
}

Call it with llm-stream "Explain TCP fast open" anthropic/claude-3.5-sonnet. For scripts, add set -e and check curl exit code separately because the pipe to python masks curl failures.

Troubleshooting

Empty output: Ensure -N is present. Without it, curl may wait for the connection to close before printing, which defeats streaming.

Malformed JSON: Some proxies inject whitespace or comment lines (: ping). Strip non-data: lines before parsing, as shown above.

Rate limits: A 429 before the first data: appears is a normal HTTP error. If you see it mid-stream, the gateway killed the connection. Retry with backoff. Automatic fallback at the gateway level masks this for many providers, but your client should still handle partial streams gracefully.

Truncated UTF-8: Tokens can split multi-byte characters across chunks. Accumulate bytes and only decode at output; the Python stdout.write approach does this safely because it writes str slices from already-decoded JSON.

Stuck connection: Some servers hold the TCP connection open after [DONE]. Curl exits when the server closes it; if you pipe to a tool that waits for EOF, you may see a delay. That is normal.

Why this matters

When you treat curl streaming chat completions command line as a first-class debug surface, you stop guessing what the SDK hides. You see exactly when tokens arrive, when the connection drops, and what the final accounting looks like. That visibility is mandatory before you wire LLM calls into a production service where a silent truncation or a missing usage field turns into a billing surprise or a broken user experience.

Tagscurlstreamingclisse

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
API integrationPiping curl streaming output into jq for parsingStep-by-step guide to piping curl streaming output into jq for parsing LLM API Server-Sent Events: strip data prefixes, filter deltas, and test locally.API integrationBuilding a Postman collection from curl LLM examplesPractical guide to convert curl to Postman collection LLM API examples with ordered steps, runnable code, and verification tips for engineers.API integrationcurl one-liners for checking LLM API rate limit headersPractical curl rate limit headers one-liners to inspect LLM API throttling, quota, and reset times across OpenAI, Anthropic, and inference providers.API integrationDebugging LLM API errors with curl -vUse curl -v to debug LLM API errors by exposing raw HTTP requests, TLS issues, and provider responses—a practical how-to for engineers integrating models.API integrationSending images to a vision model with curlStep-by-step guide to a curl vision model image request: encode images, build OpenAI-compatible JSON, post with curl, and verify multimodal LLM responses.API integrationTesting API key auth with curl before writing codeLearn how to curl test api key authentication against LLM endpoints before writing code. Step-by-step CLI checks for bearer tokens, error cases, and live calls.

All curl llm api cookbook posts →