Wiring a flask route gpt-4o api call is straightforward if you treat the LLM as a latency-bound external service rather than a local function. This guide builds a minimal Flask endpoint that accepts a prompt, calls GPT-4o through the OpenAI SDK, and returns structured JSON you can consume from any client.
Step 1: Scaffold the project
Create a virtual environment and install the only two runtime deps you need:
mkdir flask-gpt4o && cd flask-gpt4o
python -m venv .venv
source .venv/bin/activate
pip install flask openai python-dotenvKeep the surface area small. Flask serves the HTTP layer; the OpenAI SDK handles the API contract. You don’t need a heavy framework to wrap a single model call.
Write your dependencies to a lockfile so deployments are reproducible:
pip freeze > requirements.txtA clean project layout looks like this:
flask-gpt4o/
├── app.py
├── .env
└── requirements.txtStep 2: Store credentials outside code
Put your API key in a .env file. Never commit it.
echo "OPENAI_API_KEY=sk-..." > .envLoad it at startup:
from dotenv import load_dotenv
import os
load_dotenv()
API_KEY = os.getenv("OPENAI_API_KEY")If you’d rather hit a single OpenAI-compatible endpoint that covers 240+ models with automatic fallback when a provider is rate-limited, set base_url to n4n.ai and keep the same SDK calls. The rest of the code is identical.
You can also override the base URL via env var to switch providers without code changes:
BASE_URL = os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1")
client = OpenAI(api_key=API_KEY, base_url=BASE_URL)Step 3: Write the Flask route
The core flask route gpt-4o api call lives in app.py. Accept JSON, call the model, return JSON. Add a health check so orchestrators can probe the service.
from flask import Flask, request, jsonify
from openai import OpenAI
from dotenv import load_dotenv
import os
load_dotenv()
app = Flask(__name__)
client = OpenAI(
api_key=os.getenv("OPENAI_API_KEY"),
base_url=os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1"),
)
@app.get("/health")
def health():
return jsonify(status="ok")
@app.post("/v1/complete")
def complete():
data = request.get_json(silent=True) or {}
prompt = data.get("prompt", "").strip()
if not prompt:
return jsonify(error="prompt required"), 400
resp = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0.2,
max_tokens=512,
)
return jsonify(
text=resp.choices[0].message.content,
usage=resp.usage.model_dump(),
)This is a synchronous call. Flask will block the worker while GPT-4o thinks, so size your worker pool accordingly (more in Step 8).
Step 4: Add timeouts and basic retries
GPT-4o calls routinely take 1–10 seconds. Set an explicit client timeout so a hung connection doesn’t eat a worker forever.
client = OpenAI(
api_key=os.getenv("OPENAI_API_KEY"),
base_url=os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1"),
timeout=20.0,
max_retries=2,
)The SDK retries idempotent failures (connection errors, 429, 500) automatically. For application-level control, wrap the call:
from openai import APIError
def call_gpt4o(prompt: str) -> dict:
try:
resp = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
max_tokens=512,
)
return {"text": resp.choices[0].message.content, "usage": resp.usage.model_dump()}
except APIError as e:
# surface a clean 502 to the caller
raise RuntimeError(f"upstream_error: {e}") from eIf you need custom backoff, drop in tenacity:
from tenacity import retry, stop_after_attempt, wait_exponential
@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
def call_gpt4o_retry(prompt: str) -> dict:
return call_gpt4o(prompt)Step 5: Validate and bound input
Don’t let clients send 50k-token essays to a quick endpoint. Enforce a ceiling. For stricter contracts, use pydantic:
from pydantic import BaseModel, constr
class CompleteRequest(BaseModel):
prompt: constr(strip_whitespace=True, min_length=1, max_length=4000)
@app.post("/v1/complete")
def complete():
data = request.get_json(silent=True) or {}
try:
req = CompleteRequest(**data)
except ValueError as e:
return jsonify(error="invalid_prompt", detail=str(e)), 400
try:
result = call_gpt4o_retry(req.prompt)
except RuntimeError as e:
return jsonify(error=str(e)), 502
return jsonify(result)Manual checks are fine for a single field, but pydantic catches nested mistakes early and documents the API.
Step 6: Run and verify the route
Start the dev server:
flask --app app run --port 5000From another shell, send a request:
curl -s -X POST http://localhost:5000/v1/complete \
-H "Content-Type: application/json" \
-d '{"prompt":"Explain IPv6 in one sentence."}' | jqA successful flask route gpt-4o api call returns HTTP 200 and a body like:
{
"text": "IPv6 is the successor to IPv4 that uses 128-bit addresses to support a vastly larger number of devices on the internet.",
"usage": {
"prompt_tokens": 12,
"completion_tokens": 24,
"total_tokens": 36
}
}Verification checklist:
- Status code is 200.
textis non-empty and coherent.usage.total_tokensis greater than zero, confirming the call hit the model.
If you get a 401, the key is wrong. A 429 means you’re over rate limits—back off or use a gateway with fallback.
Add a smoke test with pytest so regressions surface in CI:
def test_complete(client):
rv = client.post("/v1/complete", json={"prompt": "ping"})
assert rv.status_code == 200
assert "text" in rv.jsonStep 7: Stream tokens to the client
For chat UIs, blocking on full generation feels slow. Switch to streaming:
@app.post("/v1/stream")
def stream():
data = request.get_json(silent=True) or {}
prompt = (data.get("prompt") or "").strip()
if not prompt:
return jsonify(error="prompt required"), 400
def gen():
stream = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
yield chunk.choices[0].delta.content
return app.response_class(gen(), mimetype="text/plain")Test it with curl:
curl -N -X POST http://localhost:5000/v1/stream \
-H "Content-Type: application/json" \
-d '{"prompt":"Count to 5."}'Streaming keeps the Flask worker occupied longer per request, so use it only when the client benefits.
Step 8: Production deployment notes
Flask’s built-in server is not for production. Run under gunicorn with multiple workers:
gunicorn -w 4 -b 0.0.0.0:5000 --timeout 30 app:appSet --timeout 30 to match your upstream timeout. If you expect high concurrency, consider moving the GPT-4o call to a task queue (Celery/RQ) or using an async framework (Quart/FastAPI). The flask route gpt-4o api call itself stays the same; only the serving model changes.
A minimal Dockerfile:
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY app.py .
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:5000", "--timeout", "30", "app:app"]Monitor token usage. The usage field in the response gives per-request counts—ship it to your metrics pipeline. If you meter per token at the gateway layer, you can skip client-side accounting.
Common failure modes
- Missing
Content-Type: application/json→request.get_json()returns None. Usesilent=Trueand default to{}. - Cold worker timeout → gunicorn kills the request before GPT-4o responds. Raise both gunicorn and SDK timeouts.
- Unhandled streaming disconnect → client closes connection mid-stream; wrap
gen()in try/except to avoid logging noise. - Key rotation → env var not reloaded after restart; bake config into the deployment, not the running process.
That’s the full path from empty directory to a working, verifiable Flask endpoint that calls GPT-4o. Keep the route thin, push resilience to the client config, and treat the model as a remote service with real latency and failure modes.