Streaming GPT-5 responses to the terminal in Python

Streaming GPT-5 responses terminal python is straightforward if you treat the chat completions API as a Server-Sent Events (SSE) feed rather than a blocking request. This guide builds a small CLI that opens a stream, prints tokens as they land, and exits cleanly on error or interrupt.

Step 1: Set up your environment and credentials

Create a virtual environment and install the official SDK. The OpenAI Python client works against any OpenAI-compatible endpoint, so you are not locked to one vendor.

python -m venv venv
source venv/bin/activate
pip install openai python-dotenv

Store credentials in a .env file. Never hard-code keys in source.

echo "OPENAI_API_KEY=sk-your-key" >> .env
echo "BASE_URL=https://api.openai.com/v1" >> .env

If you point BASE_URL at a gateway, the same code works without changes. For example, an OpenAI-compatible gateway such as n4n.ai exposes one endpoint that fronts 240+ models and automatically falls back when a provider is rate-limited or degraded.

Load those values early. I prefer python-dotenv over shell exports for repeatability, but either works.

import os
from dotenv import load_dotenv

load_dotenv()
API_KEY = os.environ["OPENAI_API_KEY"]
BASE_URL = os.environ.get("BASE_URL", "https://api.openai.com/v1")
MODEL = "gpt-5"

Step 2: Choose the model string and endpoint

The model identifier must match what your endpoint serves. For this walkthrough we assume the literal string "gpt-5" is available on your provider. If you use a gateway, you can pass routing hints via extra headers, but the default chat completion call stays identical.

Opinion: keep the model name in one constant. CLI tools rot when model strings are scattered across functions. If you later swap to a fine-tune or a different provider’s equivalent, you change one line.

Some gateways honor client routing directives and forward provider cache-control hints; if you need that, pass extra_headers to the client constructor or the create call. That is an endpoint-specific concern and does not affect the streaming loop.

Step 3: Write the minimal streaming call

The SDK handles the SSE parsing. Set stream=True and iterate. Each chunk carries a delta with incremental content.

from openai import OpenAI

client = OpenAI(api_key=API_KEY, base_url=BASE_URL)

def stream_reply(prompt: str):
    stream = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}],
        stream=True,
        temperature=0.7,
    )
    for chunk in stream:
        delta = chunk.choices[0].delta
        if delta and delta.content:
            yield delta.content

That generator yields strings. It does not print anything yet; separation of I/O from transport is deliberate. The chunk object also has finish_reason on the final delta and optionally usage if you request stream_options={"include_usage": True}. Ignore those during the hot loop and inspect them after iteration.

Step 4: Render tokens to the terminal without jitter

Printing line-by-line destroys the streaming illusion. Use sys.stdout.write with explicit flush, or print(tok, end="", flush=True). Disable the newline so the user sees characters immediately.

import sys

def render(tokens):
    for tok in tokens:
        sys.stdout.write(tok)
        sys.stdout.flush()
    sys.stdout.write("\n")

A common mistake is to call print(tok) which appends \n. Your terminal then scrolls per token—unreadable for code or prose.

If you want a clean experience, hide the cursor before streaming and show it after:

sys.stdout.write("\033[?25l")  # hide cursor
# ... stream ...
sys.stdout.write("\033[?25h")  # show cursor

Python may buffer stdout when not attached to a TTY. Run with python -u or set PYTHONUNBUFFERED=1 if you pipe output and still expect live tokens.

Step 5: Wrap it in a CLI with argparse

Engineers land here to build tools, not notebooks. A proper CLI takes the prompt from arguments or stdin and exposes knobs.

import argparse

def main():
    p = argparse.ArgumentParser(description="Stream GPT-5 responses to the terminal")
    p.add_argument("prompt", help="Prompt to send")
    p.add_argument("--temperature", type=float, default=0.7)
    p.add_argument("--max-tokens", type=int, default=1024)
    args = p.parse_args()

    client = OpenAI(api_key=API_KEY, base_url=BASE_URL)
    stream = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": args.prompt}],
        stream=True,
        temperature=args.temperature,
        max_tokens=args.max_tokens,
    )
    sys.stdout.write("\033[?25l")
    try:
        for chunk in stream:
            delta = chunk.choices[0].delta
            if delta and delta.content:
                sys.stdout.write(delta.content)
                sys.stdout.flush()
    finally:
        sys.stdout.write("\033[?25h\n")

Now python cli.py "Explain RAFT in one paragraph" streams output. If the prompt argument is omitted, read sys.stdin.read() so the tool composes with Unix pipelines.

Step 6: Handle errors and interrupts

Networks fail. Providers return 429 or 503. Wrap the call so a broken stream does not leave a hidden cursor or half-printed line.

try:
    for chunk in stream:
        ...
except KeyboardInterrupt:
    sys.stdout.write("\033[?25h\n")
    sys.exit(130)
except Exception as e:
    sys.stdout.write("\033[?25h\n")
    print(f"stream error: {e}", file=sys.stderr)
    sys.exit(1)

If you use a gateway with fallback, transient provider errors may be retried upstream; still, your client should fail loudly after the stream breaks. Exit code 130 on Ctrl-C matches shell convention and makes the tool script-friendly.

Step 7: Verify the stream end to end

Run the CLI with a short prompt and watch for incremental rendering. Success criteria:

1. Text appears character-by-character (or token-by-token) rather than after a delay.
2. The cursor returns and a newline is printed at the end.
3. No stack trace on clean exit (Ctrl-C yields exit code 130).
4. If you add stream_options={"include_usage": True} (supported by some endpoints), the final chunk contains usage metadata. Print it after the loop:

    usage = None
    for chunk in stream:
        if chunk.usage:
            usage = chunk.usage
        ...
    if usage:
        print(f"\n[usage] prompt={usage.prompt_tokens} completion={usage.completion_tokens}")

If you see the usage line, the stream closed gracefully and metering works. You can also verify the raw endpoint with curl before writing Python:

curl -N $BASE_URL/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5","messages":[{"role":"user","content":"hi"}],"stream":true}'

The -N flag disables curl buffering. You should see data: {...} lines immediately.

Step 8: Raw SSE without the SDK (optional)

Sometimes you cannot pull in the SDK. Use requests with stream=True and parse the SSE protocol yourself. This is more code but removes a dependency.

import requests, json

def raw_stream(prompt: str):
    headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
    body = {
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "stream": True,
    }
    with requests.post(f"{BASE_URL}/chat/completions", headers=headers, json=body, stream=True) as r:
        for line in r.iter_lines():
            if not line or not line.startswith(b"data:"):
                continue
            payload = line[5:].strip()
            if payload == b"[DONE]":
                break
            data = json.loads(payload)
            delta = data["choices"][0]["delta"]
            if "content" in delta:
                yield delta["content"]

The wire format is simple: lines prefixed data: , JSON objects, and a final data: [DONE]. Do not assume keep-alive comments are absent; ignore empty lines. Some providers send : ping comments to keep the connection alive—skip lines that do not start with data:.

Step 9: Make it a reusable module

For a real CLI tool, split transport from rendering. Put the OpenAI client behind a function that accepts base_url and api_key, and returns a generator. Then your tests can feed a fake stream.

def make_client(api_key, base_url):
    return OpenAI(api_key=api_key, base_url=base_url)

def stream_tokens(client, model, prompt, **opts):
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        stream=True,
        **opts,
    )
    for chunk in resp:
        if chunk.choices[0].delta.content:
            yield chunk.choices[0].delta.content

This keeps the terminal streaming logic portable across shells, TUI frameworks, or even a Slack bot. You can wrap stream_tokens in a unittest.mock instance that yields a list of fake deltas to test rendering without network calls.

Step 10: Performance and UX notes

Streaming GPT-5 responses terminal python is mostly about respecting the SSE contract and flushing stdout. A few practical points from shipping similar tools:

• Avoid doing blocking work inside the loop. If you need to syntax-highlight or reflow text, buffer and post-process after the stream ends, or use a separate thread with a queue.
• If the user passes --silent, suppress the cursor hide/show escape codes; they are noise in log files.
• For long outputs, consider writing to a temp file and tail -f it, but for most CLIs direct stdout is fine.
• Watch for finish_reason == "length"; it means max_tokens cut the reply. Surface that to the user instead of silently stopping.

Build the CLI with argparse, handle interrupts, and verify usage metadata if your endpoint supports it. That is all you need to ship a responsive LLM command-line tool.