Prompt caching in LangChain: cut Claude costs by 90%

LangChain prompt caching Claude cost savings are real: Anthropic bills cached input tokens at 10% of the standard rate, which can drop your bill by 90% on workloads with stable prefixes. This tutorial builds a runnable example that adds cache control to a ChatAnthropic chain and verifies the cache hits in the usage metadata.

Prerequisites

• Python 3.10 or newer
• langchain-core and langchain-anthropic installed
• An ANTHROPIC_API_KEY exported in your environment
• Familiarity with LangChain message objects (SystemMessage, HumanMessage)

Install the dependencies:

pip install "langchain>=0.2.10" "langchain-anthropic>=0.2.0"
export ANTHROPIC_API_KEY=sk-ant-...

The problem: paying for the same context repeatedly

Most LLM apps send a long system prompt, a retrieved document, or a set of few-shot examples on every request. Without caching, Claude re-tokenizes and re-bills that prefix each time. For a code-review bot with a 1,000-token policy doc, a single day of support traffic can burn millions of redundant input tokens.

from langchain_anthropic import ChatAnthropic
from langchain_core.messages import SystemMessage, HumanMessage

SYSTEM_PROMPT = ("You are a strict Python code reviewer. Reject any function "
                 "that lacks type hints or docstrings. Enforce PEP 8 spacing. " * 80)  # ~1.1k tokens

llm = ChatAnthropic(model="claude-3-5-sonnet-20240620", max_tokens=256)

def ask(question: str) -> str:
    resp = llm.invoke([SystemMessage(content=SYSTEM_PROMPT),
                       HumanMessage(content=question)])
    return resp.content

print(ask("Review: def foo(): pass"))

Check the usage after one call:

resp = llm.invoke([SystemMessage(content=SYSTEM_PROMPT),
                   HumanMessage(content="Review: def foo(): pass")])
print(resp.response_metadata["usage"])

Expected output:

{
  "input_tokens": 1120,
  "output_tokens": 48
}

Every subsequent call repeats that 1120-token charge. At Claude 3.5 Sonnet list pricing ($3 per million input tokens), 1,000 calls cost ~$3.36 for the prefix alone—before you generate a single output token.

Adding prompt caching

Anthropic exposes cache_control on a content block. LangChain forwards additional_kwargs straight to the API. Mark the system message as cacheable:

system_msg = SystemMessage(
    content=SYSTEM_PROMPT,
    additional_kwargs={"cache_control": {"type": "ephemeral"}}
)

The ephemeral type tells Claude to keep the prefix in cache for at least 5 minutes. The block must be at least 1024 tokens; our prompt clears that bar. If you use a shorter prefix, Anthropic silently ignores the hint and you pay full price.

Verify the cache is working

Run two calls against the cached system message and inspect the usage object.

first = llm.invoke([system_msg, HumanMessage(content="Review: def bar(): return 1")])
print("FIRST:", first.response_metadata["usage"])

second = llm.invoke([system_msg, HumanMessage(content="Review: def baz(): return 2")])
print("SECOND:", second.response_metadata["usage"])

Expected output:

FIRST: {
  "input_tokens": 60,
  "output_tokens": 52,
  "cache_creation_input_tokens": 1120,
  "cache_read_input_tokens": 0
}
SECOND: {
  "input_tokens": 62,
  "output_tokens": 49,
  "cache_creation_input_tokens": 0,
  "cache_read_input_tokens": 1120
}

The first call pays a 25% premium on the cached prefix (cache_creation_input_tokens). The second call reads from cache: those 1120 tokens are billed at 10% of the input rate. That is the core of langchain prompt caching claude cost savings.

Calculate the savings

Using public Claude 3.5 Sonnet pricing:

• Standard input: $3.00 / 1M tokens
• Cache write: $3.75 / 1M tokens (25% premium)
• Cache read: $0.30 / 1M tokens (90% discount)

For 1,000 calls with a 1,120-token prefix:

• No cache: 1,120 × 1,000 = 1.12M input tokens → $3.36
• With cache: 1 write ($0.0042) + 999 reads (1.119M × $0.30/M = $0.3357) → ~$0.34

A 90% reduction, matching the headline. The write premium is negligible once reuse exceeds a handful of calls.

Caching long documents or few-shot examples

You are not limited to the system prompt. Any stable prefix can be cached. Attach cache_control to a HumanMessage that carries a large retrieved doc:

doc_msg = HumanMessage(
    content="<full API specification text, ~5k tokens>",
    additional_kwargs={"cache_control": {"type": "ephemeral"}}
)
chat = [system_msg, doc_msg, HumanMessage(content="What endpoint creates a user?")]

Only one cache breakpoint is allowed per request. Place it on the last message of the stable prefix. Anything after the breakpoint is re-sent and re-billed each call. If you need multiple stable sections, concatenate them into one block; Claude caches the contiguous prefix up to the marked block.

Why latency improves too

Cached prefixes skip reparsing on Anthropic’s side, lowering time-to-first-token. For 5k-token prefixes this is often a noticeable TTFT drop because the model no longer re-processes the long context. The usage metadata does not expose latency, but in practice a hot cache turns a 800ms prefill into a sub-200ms one for large prefixes.

Monitoring in production

Tracking langchain prompt caching claude cost savings in production requires capturing the usage fields on every LLM end event. LangChain callbacks give you a clean hook:

from langchain_core.callbacks import BaseCallbackHandler

class CacheMetrics(BaseCallbackHandler):
    def on_llm_end(self, response, **kwargs):
        usage = response.llm_output.get("usage", {})
        reads = usage.get("cache_read_input_tokens", 0)
        writes = usage.get("cache_creation_input_tokens", 0)
        if reads or writes:
            print(f"cache write={writes} read={reads}")

llm = ChatAnthropic(
    model="claude-3-5-sonnet-20240620",
    max_tokens=256,
    callbacks=[CacheMetrics()]
)

Pipe these numbers into your metrics backend. A healthy cache hit rate climbs toward 100% of prefix tokens on repeated traffic.

Routing through a gateway

If you front Claude with an OpenRouter-class gateway, the same LangChain code works as long as the gateway forwards cache-control hints. For example, n4n.ai forwards provider cache-control hints and meters per-token usage, so the additional_kwargs above reach Anthropic unchanged and you get accurate cache-read line items without modifying your chain.

Full runnable script

import os
from langchain_anthropic import ChatAnthropic
from langchain_core.messages import SystemMessage, HumanMessage

assert os.environ.get("ANTHROPIC_API_KEY"), "Set ANTHROPIC_API_KEY"

SYSTEM_PROMPT = ("You are a strict Python code reviewer. Reject any function "
                 "that lacks type hints or docstrings. Enforce PEP 8 spacing. " * 80)

llm = ChatAnthropic(model="claude-3-5-sonnet-20240620", max_tokens=256)

system_msg = SystemMessage(
    content=SYSTEM_PROMPT,
    additional_kwargs={"cache_control": {"type": "ephemeral"}}
)

for i, fn in enumerate(["def foo(): pass", "def bar(): return 1", "def baz(): return 2"]):
    resp = llm.invoke([system_msg, HumanMessage(content=f"Review: {fn}")])
    print(f"Call {i}: {resp.response_metadata['usage']}")

Run it. The first call shows cache_creation_input_tokens; subsequent calls show cache_read_input_tokens.

Gotchas

• Minimum length: Cacheable blocks must exceed 1024 tokens for Claude 3.5 Sonnet. Shorter prefixes are ignored.
• Prefix strictness: Changing even one character before the cache breakpoint invalidates the cache.
• Single breakpoint: Only the last block before the variable part should carry cache_control.
• TTL: Cache entries expire 5 minutes after the last read. Busy workloads stay hot; idle ones reprime.
• SDK version: Older langchain-anthropic versions dropped additional_kwargs on system messages. Pin to >=0.2.0.

Langchain prompt caching claude cost savings depend on disciplined prefix design. Identify the stable context, mark it once, and watch the cache_read_input_tokens climb in your metrics.