LangChain makes it easy to compose LLM calls, but the resulting chains are opaque when something goes wrong. Proper langchain opentelemetry instrumentation gives you distributed traces across every model call, retriever, and tool invocation without vendoring a bespoke observability stack.
Prerequisites
- Python 3.10 or newer
langchain-openaiandlangchain-core(or a comparable recent LangChain distribution)- An OpenAI API key, or an OpenAI-compatible endpoint such as a gateway that exposes
/v1/chat/completions pipaccess to install OpenTelemetry packages
You should be able to write and run a minimal LangChain chain locally. This tutorial builds one from scratch and layers tracing on top.
Install dependencies
Install the OpenTelemetry SDK, the OTLP exporter, and the LangChain instrumentation package. The instrumentation ships in the OpenTelemetry contrib tree.
pip install opentelemetry-sdk opentelemetry-exporter-otlp \
opentelemetry-instrumentation-langchain langchain-openai langchain-coreThe ConsoleSpanExporter is included with opentelemetry-sdk, so local debugging needs no extra dependency. For production you will use OTLPSpanExporter to ship spans to a collector.
Configure the tracer provider
Set up a TracerProvider before importing LangChain components that you intend to trace. The batch processor buffers spans and exports on a background thread.
import os
os.environ["OTEL_SERVICE_NAME"] = "langchain-demo"
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
provider = TracerProvider()
provider.add_span_processor(
BatchSpanProcessor(ConsoleSpanExporter())
)
trace.set_tracer_provider(provider)Avoid the synchronous SimpleSpanProcessor in anything latency-sensitive; it blocks the calling thread on every export.
Instrument LangChain
Patch the LangChain internals with a single call. The instrumentor wraps runnables, model clients, and prompt templates.
from opentelemetry.instrumentation.langchain import LangChainInstrumentor
LangChainInstrumentor().instrument()Calling instrument() is idempotent. If you later import additional LangChain modules, the patches are already active. Do not call it inside a request handler—run it once at process startup.
Run a traced chain
Build a small chain and invoke it. If you route through n4n.ai, the OpenAI-compatible endpoint addresses 240+ models and honors client routing directives, and the spans will reflect per-token metering automatically.
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
chat = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
prompt = ChatPromptTemplate.from_template("Tell me a short joke about {topic}")
chain = prompt | chat | StrOutputParser()
output = chain.invoke({"topic": "metrics"})
print(output)Expected console output
The console exporter emits one span per chain step, nested under the implicit root. A trimmed example:
{
"name": "ChatPromptTemplate.invoke",
"attributes": { "langchain.component": "prompt" }
}
{
"name": "ChatOpenAI.invoke",
"attributes": {
"llm.system": "OpenAI",
"llm.model": "gpt-3.5-turbo",
"llm.token_count.prompt": 14,
"llm.token_count.completion": 22
}
}
{
"name": "StrOutputParser.invoke",
"attributes": { "langchain.component": "parser" }
}If you see no spans, confirm LangChainInstrumentor().instrument() executed before the chain was constructed. The patching hooks class methods at import time for some components.
Capture token usage and routing metadata
LangChain returns token usage inside the model response object. The OpenTelemetry instrumentation copies those counts into span attributes, so you can aggregate cost without writing custom callbacks.
To audit them locally, add a lightweight processor:
from opentelemetry.sdk.trace.export import SpanProcessor
class TokenAuditProcessor(SpanProcessor):
def on_end(self, span):
attrs = span.attributes or {}
if "llm.token_count.prompt" in attrs:
print(f"[{span.name}] prompt={attrs['llm.token_count.prompt']} "
f"completion={attrs['llm.token_count.completion']}")
provider.add_span_processor(TokenAuditProcessor())When a gateway performs automatic fallback when a provider is rate-limited or degraded, the resolved model may differ from the requested one. The span records the actual llm.model, so you can spot fallback events after the fact. Gateways that forward provider cache-control hints expose those as attributes too, letting you verify cache hits in the trace.
Export to a real collector
Console output does not scale. Point the exporter at an OTLP receiver over gRPC:
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
otlp = OTLPSpanExporter(endpoint="http://localhost:4317", insecure=True)
provider.add_span_processor(BatchSpanProcessor(otlp))Run a local Jaeger for testing:
docker run -p 4317:4317 -p 16686:16686 jaegertracing/all-in-oneAfter executing your script, open http://localhost:16686 and search for the ChatOpenAI.invoke operation. The trace view shows the prompt, model, and parser spans as a nested tree with latency per step. This is the fastest way to find which part of a chain is slow.
Adding custom spans
Automatic instrumentation covers LangChain primitives but not your surrounding logic. Wrap business code with a tracer to correlate user actions with LLM calls:
tracer = trace.get_tracer("my_app")
with tracer.start_as_current_span("handle_support_ticket") as root:
root.set_attribute("user.id", "42")
answer = chain.invoke({"topic": "timeouts"})Because the LangChain spans are created inside the active context, they nest under handle_support_ticket. In an async app, use async with tracer.start_as_current_span(...) and await the chain; context propagation works the same.
Instrumenting tools and agents
Tools called by agents also emit spans. Decorate a function with @tool and it appears as a child span when the agent invokes it:
from langchain_core.tools import tool
@tool
def get_weather(city: str) -> str:
"""Return the current weather for a city."""
return "sunny"
# Agent setup omitted; get_weather appears as a span named "get_weather.invoke"If you build a custom retriever, wrap its invoke in a span manually so the retrieval latency is visible next to the generation latency.
Debugging common gaps
Flattened traces (no nesting) usually mean instrument() ran after the chain was imported or constructed. Restart the process and ensure instrumentation happens at module load.
Missing token attributes indicate the model client did not return usage. Upgrade to langchain-openai >= 0.1.0, which populates usage_metadata on the result.
If spans never reach Jaeger, check the collector endpoint and that the OTLP port (4317) is exposed. The gRPC exporter fails silently on connection refusal unless you configure a timeout.
Wrapping up
langchain opentelemetry instrumentation turns a black-box chain into a queryable trace graph. You get model latency, token counts, and routing metadata with a few lines of setup. From there, standard OTel tooling handles dashboards and alerts—no proprietary SDK required.