VCR cassette testing for LLM APIs solves a real problem: your integration tests shouldn’t depend on a live model endpoint, rate limits, or variable completions. By recording the first real call and replaying it offline, you get fast, deterministic CI runs without hand-writing mocks for every endpoint shape.
Prerequisites
- Python 3.10 or newer
openaiPython client (v1.x)vcrpy(v4.4+) andpytest- A valid API key for the initial recording run only
- Basic comfort with pytest fixtures
If you are on an older Python, the code below will need minor adjustments; vcrpy relies on modern httpx/urllib3 internals.
Install dependencies
pip install "openai>=1.0" "vcrpy>=4.4" pytest pyyamlWe pin major versions because the OpenAI client changed its request shape significantly pre-1.0.
Record a real call
Create record.py. The first run hits the network; subsequent runs replay the cassette if record_mode="once" and the file exists.
import vcr
import openai
# If you record against a gateway such as n4n.ai—one OpenAI-compatible
# endpoint covering 240+ models with automatic fallback—you get a stable
# interface instead of a single provider's quirks.
client = openai.OpenAI(api_key="sk-real-key")
with vcr.use_cassette(
"fixtures/cassettes/chat.yaml",
record_mode="once",
filter_headers=[("authorization", "REDACTED")],
match_on=["method", "host", "path", "body"],
):
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Say hello in one word."}],
temperature=0,
)
print("RESPONSE:", resp.choices[0].message.content)
print("TOKENS:", resp.usage.total_tokens)Run it:
python record.pyExpected output (real call, so wording may differ):
RESPONSE: Hello.
TOKENS: 11The cassette is written to fixtures/cassettes/chat.yaml. Open it. You’ll see a YAML list of interactions with request/response bodies. The authorization header is already replaced with REDACTED.
interactions:
- request:
method: POST
uri: https://api.openai.com/v1/chat/completions
headers:
authorization: REDACTED
body: '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say hello
in one word."}],"temperature":0}'
response:
status: 200
headers:
content-type: application/json
body:
string: '{"id":"chatcmpl-...","choices":[{"message":{"content":"Hello."}}],"usage":{"total_tokens":11}}'That’s the core of vcr cassette testing llm api workflows: a recorded HTTP contract.
Pin the response for determinism
Models are non-deterministic even at temperature=0. To make tests reproducible, sanitize the response body before it’s saved, or edit the YAML directly. Programmatic sanitization is better because it survives re-recording.
import json
def before_record_response(response):
body = response["body"]["string"].decode("utf-8")
data = json.loads(body)
data["choices"][0]["message"]["content"] = "Hello."
data["usage"] = {"prompt_tokens": 10, "completion_tokens": 1, "total_tokens": 11}
response["body"]["string"] = json.dumps(data).encode("utf-8")
return responsePass it into the cassette context:
with vcr.use_cassette(
"fixtures/cassettes/chat.yaml",
record_mode="once",
filter_headers=[("authorization", "REDACTED")],
before_record_response=before_record_response,
match_on=["method", "host", "path", "body"],
):
...Delete the old cassette and re-run record.py. Now the file contains exactly your pinned string.
Write a pytest test that replays
In test_llm.py, force record_mode="none" so the test fails if it tries to hit the network—exactly what you want in CI.
import vcr
import openai
@vcr.use_cassette(
"fixtures/cassettes/chat.yaml",
record_mode="none",
filter_headers=[("authorization", "REDACTED")],
match_on=["method", "host", "path", "body"],
)
def test_chat_completion_deterministic():
client = openai.OpenAI(api_key="fake-not-real")
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Say hello in one word."}],
temperature=0,
)
assert resp.choices[0].message.content == "Hello."
assert resp.usage.total_tokens == 11Run:
pytest test_llm.py -qExpected:
.
1 passed in 0.12sNo network calls were made. The test matches the request shape against the cassette and returns the stored response.
Match tightly or loosely
The match_on tuple controls replay matching. For LLM calls, matching on body is strict: if you change the prompt or add a parameter, the cassette won’t match and VCR raises CannotOverwriteExistingCassetteException (in once mode) or falls back to record (in new_episodes). In CI with none, it fails the test, which is correct—you changed the contract.
If you want to ignore temperature or seed, use filter_post_data_parameters (for form-encoded) or a custom before_record_request to mutate the body JSON before matching. Example:
def before_record_request(request):
import json
if "chat/completions" in request.uri:
body = json.loads(request.body)
body.pop("temperature", None)
request.body = json.dumps(body).encode()
request.headers["content-length"] = str(len(request.body))
return requestStreaming responses
VCRpy records the raw socket stream poorly. If your code uses stream=True, either disable streaming during tests or mock at a higher level. Practical approach:
resp = client.chat.completions.create(..., stream=False)Record that, then in tests assert the non-stream shape. If you must test streaming logic, fake the iterator with a local fixture rather than a cassette. VCR cassette testing llm api streams is an edge case not worth the pain for most teams.
Async and multiple interactions
If you use AsyncOpenAI, wrap the coroutine in the same cassette context. VCRpy intercepts httpx under the hood, so it works transparently.
import vcr
import openai
import asyncio
async def main():
client = openai.AsyncOpenAI(api_key="fake")
with vcr.use_cassette("fixtures/cassettes/chat.yaml", record_mode="none"):
resp = await client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Say hello in one word."}],
temperature=0,
)
return resp.choices[0].message.content
print(asyncio.run(main()))For multi-turn conversations, record a sequence: the cassette stores ordered interactions and replays them in order. Keep match_on strict; if you branch logic based on prior response, the replay will diverge. In those cases, split into separate cassettes per path.
Use JSON cassettes for cleaner diffs
YAML is default but noisy. Switch to JSON:
vcr.use_cassette("fixtures/cassettes/chat.json", serializer="json", record_mode="none")JSON makes code review of recorded tokens straightforward and avoids whitespace churn in PRs.
Wire into CI
Use a conftest.py to centralize config and switch record mode via environment variable.
import os
import vcr
import pytest
@pytest.fixture
def vcr_config():
return {
"record_mode": os.environ.get("VCR_RECORD", "none"),
"filter_headers": [("authorization", "REDACTED")],
"match_on": ["method", "host", "path", "body"],
}Then decorate tests with @vcr.use_cassette("fixtures/cassettes/chat.yaml") without repeating options. In GitHub Actions, run normally (replay). To refresh cassettes, run a job with VCR_RECORD=once and commit the updated file.
When cassettes rot
Models get deprecated. If gpt-4o-mini disappears, your cassette still replays the old response, which may hide a real breaking change. Mitigate by periodically re-recording with VCR_RECORD=all and reviewing diffs. Also keep a separate smoke test that hits a cheap real endpoint (or a gateway with fallback) to catch auth or schema drift.
The vcr cassette testing llm api pattern is not a replacement for eval suites; it’s a guard for your integration code—retries, parsing, error handling—not for model quality.
Final checklist
- Record once with real key, redact headers.
- Pin response bodies for determinism.
- Set
record_mode="none"in CI. - Match on body, but trim volatile fields.
- Keep cassettes in version control; treat them as contracts.
- Prefer JSON serialization for reviewability.
That’s the whole loop. Your LLM integration tests now run in milliseconds, offline, and fail loudly when your request shape changes.