Mocking OpenAI API responses testing is a prerequisite for any serious LLM app development that needs fast, deterministic, and offline test cycles. This guide walks through concrete steps to stub the Chat Completions endpoint, intercept SDK calls, and validate your logic without burning tokens or waiting on rate limits.
Step 1: Choose your interception layer
Three practical places exist to fake the API: a standalone stub HTTP server, SDK method patching, or request interception inside your test runner.
A stub server gives the highest fidelity. Your code makes real TCP calls to localhost, so retries, timeouts, and streaming all execute as they would in production. SDK patching (monkeypatching client.chat.completions.create) is faster but bypasses the transport layer entirely. Request interception with respx or msw sits in between: it mocks at the HTTP library level without a separate process.
Use a stub server for integration tests. Use interception for unit tests where you only care about the parsed object.
If you want to record real traffic once and replay it, vcr.py (Python) or polly.js (JS) can cache live responses to YAML cassettes. That works, but cassettes rot when the API schema changes. I prefer hand-written fixtures for core logic.
Step 2: Stand up a local stub server
Flask is enough. The stub below mirrors the 2023-10-01-preview response shape for a non-streaming chat completion.
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.route("/v1/chat/completions", methods=["POST"])
def chat_completions():
body = request.json
return jsonify({
"id": "chatcmpl-mock",
"object": "chat.completion",
"created": 1690000000,
"model": body.get("model", "gpt-3.5-turbo"),
"choices": [{
"index": 0,
"message": {
"role": "assistant",
"content": "This is a mocked response."
},
"finish_reason": "stop"
}],
"usage": {
"prompt_tokens": 5,
"completion_tokens": 5,
"total_tokens": 10
}
})
if __name__ == "__main__":
app.run(port=4000)Run it:
pip install flask
python stub.pyYou now have http://localhost:4000/v1/chat/completions. To simulate a degraded provider, return a 429:
@app.route("/v1/chat/completions", methods=["POST"])
def chat_completions():
if request.headers.get("X-Simulate") == "rate-limit":
return jsonify({"error": "rate limited"}), 429
# ... normal mockThat lets you test your fallback logic locally.
Step 3: Point the OpenAI SDK at the stub
Set base_url via environment variable so production code stays unchanged.
import os
import openai
client = openai.OpenAI(
base_url=os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1"),
api_key=os.getenv("OPENAI_API_KEY", "sk-mock")
)
resp = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hello"}]
)
assert resp.choices[0].message.content == "This is a mocked response."In your .env.test:
OPENAI_BASE_URL=http://localhost:4000/v1
OPENAI_API_KEY=sk-mockVerification: running the snippet prints the mocked string and exits 0.
Step 4: Mock streaming responses
Streaming is where many bugs hide. The stub must emit Server-Sent Events with data: prefixes and a final [DONE].
from flask import Response, stream_with_context
import json
@app.route("/v1/chat/completions", methods=["POST"])
def chat_completions():
body = request.json
if body.get("stream"):
def gen():
chunks = ["This ", "is ", "a ", "mocked ", "stream."]
for i, c in enumerate(chunks):
data = {
"id": "chatcmpl-mock",
"object": "chat.completion.chunk",
"created": 1690000000,
"model": body.get("model"),
"choices": [{
"index": 0,
"delta": {"content": c},
"finish_reason": None if i < len(chunks)-1 else "stop"
}]
}
yield f"data: {json.dumps(data)}\n\n"
yield "data: [DONE]\n\n"
return Response(stream_with_context(gen()), mimetype="text/event-stream")
# ... non-stream branchClient:
stream = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Hi"}],
stream=True
)
out = "".join(
chunk.choices[0].delta.content or ""
for chunk in stream
)
assert out == "This is a mocked stream."If your app uses stream_options: {"include_usage": True}, add a final chunk with "usage": {...} and empty choices.
Step 5: Intercept SDK calls in tests
For Python, respx mocks httpx (the transport used by the official SDK).
import respx
import openai
import json
@respx.mock
def test_completion():
route = respx.post("https://api.openai.com/v1/chat/completions").mock(
return_value=respx.MockResponse(
json={
"id": "chatcmpl-test",
"object": "chat.completion",
"created": 1,
"model": "gpt-3.5-turbo",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": "ok"},
"finish_reason": "stop"
}],
"usage": {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}
},
status_code=200
)
)
client = openai.OpenAI(api_key="sk-test")
resp = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "test"}]
)
assert resp.choices[0].message.content == "ok"
assert route.calledFor TypeScript, msw intercepts fetch:
import { setupServer } from 'msw/node'
import { http, HttpResponse } from 'msw'
const server = setupServer(
http.post('https://api.openai.com/v1/chat/completions', () => {
return HttpResponse.json({
id: 'chatcmpl-ts',
object: 'chat.completion',
created: 1,
model: 'gpt-4',
choices: [{ index: 0, message: { role: 'assistant', content: 'mocked' }, finish_reason: 'stop' }],
usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 }
})
})
)
server.listen({ onUnhandledRequest: 'error' })Run with vitest or jest. The test passes when the parsed content equals "mocked".
Step 6: Cover function calls, embeddings, and vision
Mocking OpenAI API responses testing must extend beyond plain text. For tool calls:
{
"choices": [{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"tool_calls": [{
"id": "call_abc",
"type": "function",
"function": {"name": "get_weather", "arguments": "{\"loc\":\"SF\"}"}
}]
},
"finish_reason": "tool_calls"
}]
}Your app should parse tool_calls and dispatch to the local function. Write a test that asserts the right function name and arguments are extracted.
Embeddings stub:
@app.route("/v1/embeddings", methods=["POST"])
def embeddings():
return jsonify({
"object": "list",
"data": [{"object": "embedding", "index": 0, "embedding": [0.1, 0.2, 0.3]}],
"model": "text-embedding-3-small",
"usage": {"prompt_tokens": 3, "total_tokens": 3}
})Vision requests send base64 images. Mock the response exactly as a text completion; the client-side image encoding is your code’s responsibility, so you don’t need to fake the image itself.
Step 7: Verify success and wire into CI
A green test is necessary but not sufficient. Add a smoke test that boots the stub, calls your service’s top-level function, and checks the side effect (DB write, UI state, downstream message).
pytest tests/test_mock_openai.py --verboseExpected:
tests/test_mock_openai.py::test_completion PASSED
tests/test_mock_openai.py::test_stream PASSED
tests/test_mock_openai.py::test_tool_call PASSEDIn GitHub Actions:
- name: Start stub
run: python stub.py &
- name: Run tests
run: pytestIf you later need to run the same code against real models with provider fallback, point base_url at an OpenAI-compatible gateway such as n4n.ai, which exposes one endpoint for 240+ models and honors cache-control headers. Keep the mock layer for unit tests; use the live gateway only in staging.
Step 8: Avoid leaky mocks
Reset state after each test. With respx, the context manager does this. With msw:
afterEach(() => server.resetHandlers())
afterAll(() => server.close())Never hardcode a real key in test files. The mock client accepts any string; enforce OPENAI_API_KEY is unset or dummy in CI secrets.
Treat fixtures as contract tests. When the real API adds a field like system_fingerprint, add it to your mock. If your parser breaks on the new field, the test should fail locally, not in production.
Mocking OpenAI API responses testing is not glamorous, but it is the difference between a flaky LLM build and one you can refactor at 2 a.m. Build the stub once, reuse it across the suite, and your tests will run in milliseconds instead of seconds per token.