Debugging LCEL chains with LangSmith tracing

LangChain Expression Language (LCEL) chains are powerful but opaque — when a chain fails or produces unexpected output, you need visibility into every step. LangSmith tracing gives you that visibility. This guide walks through instrumenting an LCEL chain, capturing traces, and diagnosing common failure modes, with code you can run today.

Step 1: Install dependencies and configure credentials

Start with a clean environment. You need LangChain core, the OpenAI integration (or your provider of choice), and the LangSmith SDK.

pip install -q langchain-core langchain-openai langsmith

Set your LangSmith API key and project name. You can get an API key from the LangSmith settings page.

export LANGCHAIN_API_KEY="lsv2_pt_..."
export LANGCHAIN_PROJECT="lcel-debugging-demo"
export LANGCHAIN_TRACING_V2="true"

The LANGCHAIN_TRACING_V2=true flag enables the v2 tracer, which captures LCEL chain execution automatically. Without it, you only get top-level run metadata.

Verify the setup works:

from langsmith import Client

client = Client()
print(client.list_projects())

You should see your project listed. If you get an authentication error, double-check the API key.

Step 2: Build a representative LCEL chain

Create a chain that exercises the features you’ll actually debug: prompt templating, model invocation, output parsing, and a conditional branch. This example classifies a support ticket and routes it to a specialized handler.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableBranch, RunnableLambda, RunnablePassthrough
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

classify_prompt = ChatPromptTemplate.from_template(
    "Classify this support ticket as 'billing', 'technical', or 'account'. "
    "Respond with only the category.\n\nTicket: {ticket}"
)

classify_chain = classify_prompt | llm | StrOutputParser()

billing_prompt = ChatPromptTemplate.from_template(
    "You are a billing specialist. Respond to: {ticket}"
)
technical_prompt = ChatPromptTemplate.from_template(
    "You are a technical support engineer. Respond to: {ticket}"
)
account_prompt = ChatPromptTemplate.from_template(
    "You are an account manager. Respond to: {ticket}"
)

billing_chain = billing_prompt | llm | StrOutputParser()
technical_chain = technical_prompt | llm | StrOutputParser()
account_chain = account_prompt | llm | StrOutputParser()

route = RunnableBranch(
    (lambda x: "billing" in x["category"].lower(), billing_chain),
    (lambda x: "technical" in x["category"].lower(), technical_chain),
    (lambda x: "account" in x["category"].lower(), account_chain),
    RunnableLambda(lambda x: "Unable to route ticket"),
)

full_chain = (
    {"category": classify_chain, "ticket": RunnablePassthrough()}
    | route
)

This chain uses RunnableBranch for conditional routing — a common pattern that’s hard to debug without traces because the branch taken isn’t visible in the final output.

Step 3: Run the chain and inspect the trace

Execute the chain with a test input. The v2 tracer automatically creates a run in LangSmith.

test_ticket = "I was charged twice for my subscription last month. Can you refund the duplicate?"
result = full_chain.invoke(test_ticket)
print(result)

Open the LangSmith UI at https://smith.langchain.com and navigate to your project. You’ll see a run named after the chain’s entry point (typically “RunnableSequence”). Click it to open the trace view.

The trace shows:

• Timeline: Wall-clock time for each runnable
• Inputs/outputs: Exact data flowing between steps
• Token usage: Per-model token counts
• Errors: Stack traces if any step fails

Verification: The trace should show four top-level spans — classify, then one of the three specialist chains. Confirm the category classification matches your expectation.

Step 4: Add custom metadata and tags for filtering

Production chains process thousands of requests. You need to filter traces by user, session, feature flag, or experiment group. Attach metadata at invocation time using the config parameter.

result = full_chain.invoke(
    test_ticket,
    config={
        "metadata": {
            "user_id": "user_12345",
            "session_id": "sess_abcde",
            "feature_flag": "new_routing_v2",
        },
        "tags": ["support", "routing", "v2"],
    }
)

In the LangSmith UI, you can now filter by user_id:user_12345 or tag:support. This is essential when debugging a specific user’s complaint.

Verification: Refresh the trace list and filter by the tag “support”. Only your test run should appear.

Step 5: Debug a classification error

Let’s simulate a real debugging scenario. The classifier mislabels a ticket, sending it to the wrong handler.

ambiguous_ticket = "My API key stopped working after I updated my billing info"
result = full_chain.invoke(ambiguous_ticket, config={"tags": ["debug"]})
print(result)

Check the trace. Look at the classify_chain span — specifically its output. You’ll likely see “technical” or “account” when the billing context matters more.

The fix: improve the classification prompt with few-shot examples.

classify_prompt_v2 = ChatPromptTemplate.from_messages([
    ("system", "Classify support tickets as 'billing', 'technical', or 'account'. "
     "Prioritize billing when payment/charge language appears."),
    ("human", "Ticket: I was charged twice for my subscription\nCategory: billing"),
    ("human", "Ticket: My API key returns 401\nCategory: technical"),
    ("human", "Ticket: I need to change my email address\nCategory: account"),
    ("human", "Ticket: {ticket}\nCategory:"),
])

classify_chain_v2 = classify_prompt_v2 | llm | StrOutputParser()

full_chain_v2 = (
    {"category": classify_chain_v2, "ticket": RunnablePassthrough()}
    | route
)

result = full_chain_v2.invoke(ambiguous_ticket, config={"tags": ["debug", "v2"]})
print(result)

Compare the two traces side by side in LangSmith. The v2 trace should show “billing” classification and route to billing_chain.

Verification: The trace for v2 shows the correct branch taken. The output references refunds or charges, not API troubleshooting.

Step 6: Inspect streaming and intermediate outputs

LCEL chains support streaming. When debugging latency or partial failures, you need to see intermediate tokens. Use astream_events with the v2 tracer to capture streaming events.

import asyncio

async def stream_debug():
    async for event in full_chain_v2.astream_events(
        "I want to cancel my subscription and get a refund",
        config={"tags": ["stream-debug"]},
        version="v2",
    ):
        kind = event["event"]
        name = event.get("name", "")
        data = event.get("data", {})
        
        if kind == "on_chain_start":
            print(f"▶ {name}: {data.get('input', '')[:80]}")
        elif kind == "on_chain_end":
            print(f"■ {name}: {str(data.get('output', ''))[:80]}")
        elif kind == "on_chat_model_stream":
            chunk = data.get("chunk")
            if chunk and chunk.content:
                print(f"  ⟳ {name}: {chunk.content}", end="", flush=True)
        elif kind == "on_chain_error":
            print(f"✗ {name}: {data.get('error')}")

asyncio.run(stream_debug())

The trace captures each streaming chunk as a child span under the model invocation. This lets you correlate latency spikes with specific token positions.

Verification: Output shows streaming tokens arriving incrementally. In LangSmith, the model span has child spans for each chunk (or batched chunks).

Step 7: Debug parallel branches and fan-out

LCEL’s RunnableParallel (the | operator with dicts) executes branches concurrently. A common bug: one branch fails silently because its error is swallowed or logged separately.

from langchain_core.runnables import RunnableParallel

# Simulate a chain that enriches ticket with user data in parallel
enrich_chain = RunnableParallel(
    user_profile=RunnableLambda(lambda x: {"plan": "pro", "tickets_this_month": 3}),
    recent_tickets=RunnableLambda(lambda x: [{"id": 1, "topic": "billing"}]),
    sentiment=RunnableLambda(lambda _: 1/0),  # Intentional error
)

debug_chain = enrich_chain | RunnableLambda(lambda x: f"Enriched: {x}")

try:
    debug_chain.invoke("test", config={"tags": ["parallel-debug"]})
except Exception as e:
    print(f"Caught: {e}")

Open the trace. You’ll see enrich_chain with three child spans — two successful, one errored. The error span shows the ZeroDivisionError with full stack trace. The parent enrich_chain span is marked as errored.

Verification: The trace shows all three parallel branches. The failed branch has a red error indicator with the exception details. The other two branches show their outputs normally.

Step 8: Use LangSmith’s evaluation workflow for regression testing

Debugging isn’t just reactive. Set up a dataset in LangSmith to catch regressions when you modify prompts or routing logic.

from langsmith import Client

client = Client()

# Create a dataset if it doesn't exist
dataset_name = "support-routing-golden-set"
try:
    dataset = client.read_dataset(dataset_name=dataset_name)
except:
    dataset = client.create_dataset(dataset_name=dataset_name)

# Add examples
examples = [
    {"inputs": {"ticket": "Double charge on my card"}, "outputs": {"expected_category": "billing"}},
    {"inputs": {"ticket": "API returning 500 errors"}, "outputs": {"expected_category": "technical"}},
    {"inputs": {"ticket": "Change my password"}, "outputs": {"expected_category": "account"}},
]

client.create_examples(
    dataset_id=dataset.id,
    examples=examples,
)

print(f"Dataset ready: {dataset.id}")

Now run an evaluation against your chain:

from langchain.smith import RunEvalConfig, run_on_dataset

eval_config = RunEvalConfig(
    evaluators=[
        RunEvalConfig.Criteria("correctness"),
        RunEvalConfig.Criteria("helpfulness"),
    ],
    custom_evaluators=[],
)

run_on_dataset(
    client=client,
    dataset_name=dataset_name,
    llm_or_chain_factory=full_chain_v2,
    evaluation=eval_config,
    project_name="lcel-debugging-eval",
)

The evaluation runs each example through your chain and scores outputs against criteria. Failed evaluations appear in the project with detailed comparison views.

Verification: Open the evaluation project. Each example shows the chain trace alongside the expected output and evaluator scores. Red rows indicate regressions.

Step 9: Export traces for offline analysis

Sometimes you need to share traces with teammates who don’t have LangSmith access, or analyze patterns across thousands of runs programmatically.

# Export runs to JSONL for local analysis
runs = client.list_runs(project_name="lcel-debugging-demo", filter='tags:"debug"')

with open("debug_traces.jsonl", "w") as f:
    for run in runs:
        f.write(run.json() + "\n")

print(f"Exported {sum(1 for _ in open('debug_traces.jsonl'))} traces")

Each line is a complete run object with inputs, outputs, timing, token usage, and the full span tree. You can load this into pandas or a notebook for cohort analysis.

import pandas as pd
import json

records = [json.loads(line) for line in open("debug_traces.jsonl")]
df = pd.DataFrame(records)

# Analyze latency by chain component
df["latency_ms"] = (pd.to_datetime(df["end_time"]) - pd.to_datetime(df["start_time"])).dt.total_seconds() * 1000
print(df.groupby("name")["latency_ms"].describe())

Verification: The JSONL file contains valid run objects. The pandas analysis shows latency distributions per runnable name.

Step 10: Configure sampling for high-volume production

You can’t trace every request at scale. LangSmith supports probabilistic sampling via the LANGCHAIN_SAMPLING_RATE environment variable (0.0 to 1.0).

export LANGCHAIN_SAMPLING_RATE="0.1"  # Trace 10% of requests

For more control, implement custom sampling in your application code:

import random
import os

SAMPLE_RATE = float(os.getenv("TRACE_SAMPLE_RATE", "0.05"))

def should_trace(user_id: str) -> bool:
    # Deterministic sampling by user for consistent debugging
    return hash(user_id) % 100 < SAMPLE_RATE * 100

# In your request handler:
config = {}
if should_trace(user_id):
    config["metadata"] = {"user_id": user_id, "sampled": True}
    config["tags"] = ["sampled"]
else:
    config["metadata"] = {"user_id": user_id, "sampled": False}

result = full_chain_v2.invoke(ticket, config=config)

This gives you consistent sampling per user — if a user reports an issue, their requests are either all traced or none, making investigation straightforward.

Verification: Send 100 requests with different user IDs. Approximately 5% should appear in LangSmith with the “sampled” tag.

Common failure modes and their trace signatures

What to do next

You now have a complete debugging workflow: instrument, run, inspect, iterate. The key habits:

1. Always run with LANGCHAIN_TRACING_V2=true in development and staging
2. Tag every invocation with user/session context — future you will thank you
3. Compare traces side-by-side when iterating on prompts or routing logic
4. Build golden datasets for regression testing before deploying chain changes
5. Sample in production — 5-10% with deterministic user-based sampling

The v2 tracer captures LCEL’s lazy execution faithfully. What you see in the trace is what actually ran — no sampling gaps, no missing intermediate states. Use it.