Setting up LangSmith tracing for LangChain apps

Getting langsmith tracing langchain setup right takes more than flipping a boolean. In this tutorial we wire up end-to-end observability for a LangChain app using environment variables and the native tracing client, then verify that runs appear with full latency and token breakdowns. You will leave with a pattern that works in both scripts and production services.

Prerequisites

• Python 3.9 or newer
• pip install langchain langchain-openai langsmith python-dotenv
• A LangSmith API key from your workspace settings
• An OpenAI API key, or access to any OpenAI-compatible inference endpoint

If you have not created a LangSmith project, the platform defaults to “default”. Do not use that for real work—isolate environments explicitly via LANGCHAIN_PROJECT.

Step 1: Configure environment variables

LangChain reads tracing configuration from the process environment at import time. Put these in a .env file at your project root:

LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=ls__your_key_here
LANGCHAIN_PROJECT=langchain-dev
OPENAI_API_KEY=sk-your_openai_key

Load them before any LangChain import:

from dotenv import load_dotenv
load_dotenv()

import os
assert os.environ.get("LANGCHAIN_TRACING_V2") == "true"
assert os.environ.get("LANGCHAIN_PROJECT") == "langchain-dev"

The LANGCHAIN_PROJECT variable groups runs. I prefer per-environment naming (langchain-dev, langchain-prod) rather than the generic default. The v2 flag is required; v1 tracing is deprecated and missing token metadata.

Step 2: Build and run a minimal chain

Use the current langchain-openai package, not the deprecated langchain.chat_models module. The LCEL pipe syntax is the cleanest way to compose a prompt and model.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

model = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a terse senior engineer."),
    ("user", "Explain {concept} in one sentence.")
])
chain = prompt | model

response = chain.invoke({"concept": "LangSmith tracing"})
print(response.content)

Expected console output (model wording may vary):

LangSmith tracing captures each step of a LangChain run as a nested span for debugging.

Open the LangSmith UI under the langchain-dev project. You will see a root run named RunnableSequence with two child spans: one for ChatPromptTemplate and one for ChatOpenAI. Latency, token usage, and the serialized inputs/outputs are attached automatically. This is the baseline proof that your langsmith tracing langchain setup is live.

Step 3: Confirm traces programmatically

Do not trust the UI alone—query the API to assert runs landed. The langsmith client is synchronous by default.

from langsmith import Client

client = Client()
runs = client.list_runs(project_name="langchain-dev", limit=1)
run = next(runs)
print(run.name, run.status, run.total_tokens)

Expected output (IDs differ):

RunnableSequence success 54

If total_tokens is None, your model integration did not report usage. Upgrade to langchain-openai>=0.0.8 and langsmith>=0.1.0. Token counts are non-negotiable for cost debugging.

Step 4: Add custom spans with @traceable

Library code often needs its own spans outside the LCEL graph. The @traceable decorator from langsmith keeps domain logic clean and nests correctly under the calling run.

from langsmith import traceable

@traceable(name="retrieve_docs")
def fake_retriever(query: str) -> list[str]:
    # pretend this hits a vector store
    return [f"doc chunk for {query}"]

@traceable(name="augment")
def build_context(chunks: list[str]) -> str:
    return "\n".join(chunks)

def rag_answer(q: str):
    docs = fake_retriever(q)
    ctx = build_context(docs)
    return chain.invoke({"concept": ctx[:20]})

print(rag_answer("tracing"))

In LangSmith, rag_answer becomes a parent run with retrieve_docs, augment, and the chain as children. This hierarchy is the fastest way to separate a slow retriever from a slow model call.

Step 5: Capture exceptions

Tracing is most valuable when things break. Let the exception propagate; LangSmith records the error type, message, and stack trace.

@traceable(name="boom")
def faulty_call():
    raise ValueError("provider returned 429")

try:
    faulty_call()
except ValueError as e:
    print("caught:", e)

The run shows status: error and the exception message. Set an alert in LangSmith on error status to catch degraded providers in production without polling dashboards.

Step 6: Route through an OpenAI-compatible gateway

If you proxy model traffic through a gateway, tracing still works as long as the client speaks the OpenAI shape. For example, n4n.ai exposes a single OpenAI-compatible endpoint covering 240+ models and forwards provider cache-control hints; point ChatOpenAI at its base URL and LangSmith captures the same spans without code changes.

model = ChatOpenAI(
    model="anthropic/claude-3-haiku",
    base_url="https://api.n4n.ai/v1",
    api_key=os.environ["N4N_API_KEY"],
    temperature=0,
)

No callback wiring is required. The total_tokens field populates from the gateway response, so per-token metering in LangSmith matches what the gateway reports. This keeps your langsmith tracing langchain setup agnostic to which backend serves the tokens.

Step 7: Production hardening

A few opinionated practices that pay off:

Project per environment

Never share a project between dev and prod. Inject LANGCHAIN_PROJECT from your deploy config. Mixed data makes latency baselines useless.

Attach metadata

chain.invoke(
    {"concept": "metadata"},
    config={"metadata": {"user_id": "u_123", "feature": "docs"}}
)

Metadata is filterable in the UI and via client.list_runs(metadata={"feature": "docs"}).

Sample high-volume traffic

At scale, full tracing on every request gets expensive. Use sampling:

os.environ["LANGCHAIN_SAMPLING_RATE"] = "0.1"  # trace 10%

Redact PII

LangSmith stores prompt and output text. Strip sensitive fields before invoke if your compliance window requires it.

Step 8: Async and streaming traces

LangSmith handles async natively. Use ainvoke and the spans still nest:

import asyncio

async def main():
    result = await chain.ainvoke({"concept": "async tracing"})
    print(result.content)

asyncio.run(main())

Streaming also traces correctly—each chunk is not a separate run, but the parent ChatOpenAI run shows stream: true and final token counts after completion. Do not wrap streaming in your own loops expecting per-token spans; that creates noise.

Expected final trace tree

After running the RAG example with a gateway model, your LangSmith project should show:

rag_answer (success)
├── retrieve_docs (success)
├── augment (success)
└── RunnableSequence (success)
    ├── ChatPromptTemplate (success)
    └── ChatOpenAI (success, 32 tokens)

That tree is the payoff for a correct langsmith tracing langchain setup: you see exactly where latency and token cost accumulate.

Troubleshooting

• No runs appearing: confirm LANGCHAIN_TRACING_V2 is literally "true", not 1 or True.
• Empty token counts: upgrade langchain-openai and langsmith to current releases.
• Project not found: the project is created on first run; check spelling and environment scoping.
• Decorator not tracing: ensure @traceable wraps a function called inside a traced context, or it creates a root run.

Set this configuration once, and every future LangChain app inherits observability by default.