Tracing LlamaIndex agent runs with OpenTelemetry

Tracing LlamaIndex agents with OpenTelemetry turns a black-box ReAct loop into a navigable tree of spans for each LLM call, tool invocation, and retrieval. Without it, you are left grepping logs to figure out why an agent burned 40k tokens or called the wrong function.

Step 1: Install the required packages

Use Python 3.10+. Install LlamaIndex core, the OpenAI agent package, the OpenTelemetry SDK, the LlamaIndex OTel bridge, and an exporter.

pip install llama-index-core llama-index-agents-openai llama-index-instrumentation-opentelemetry opentelemetry-sdk opentelemetry-exporter-otlp

If you use a specific LLM, install its integration (e.g., llama-index-llms-openai). For local dev we will use the OpenAI-compatible HTTP interface so the same code works against any compliant gateway.

Step 2: Initialize the OpenTelemetry SDK

Configure a tracer provider and exporters before importing any LlamaIndex agent code. The OTLP exporter pushes spans to any collector (Jaeger, Tempo, Honeycomb). For local verification, the console exporter prints spans to stdout.

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(ConsoleSpanExporter()))
provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")
    )
)
trace.set_tracer_provider(provider)

The console exporter is enough to prove instrumentation works. The OTLP line is what you keep in staging and production.

Step 3: Wire the LlamaIndex OpenTelemetry span handler

LlamaIndex dispatches events through a global instrumentation registry. Attach the OTel span handler so those events become OTel spans. This is the core of tracing LlamaIndex agents with OpenTelemetry.

from llama_index.core.instrumentation import get_dispatcher
from llama_index.instrumentation.opentelemetry import OpenTelemetrySpanHandler

dispatcher = get_dispatcher()
dispatcher.add_span_handler(OpenTelemetrySpanHandler())

One registration captures every subsequent query engine, chat engine, and agent step. You do not need to decorate your tools manually.

Caveat: handler ordering

The dispatcher is global. Register the handler once at process startup. If you add it inside a request handler in a web app, you will duplicate span handlers and double-emit spans, skewing durations and cost attributes.

Step 4: Configure the LLM and build an agent

Set the model. If you want one OpenAI-compatible endpoint that addresses 240+ models with automatic fallback when a provider is rate-limited, point LlamaIndex at n4n.ai’s gateway instead of hard-coding a vendor base URL.

from llama_index.core import Settings
from llama_index.llms.openai import OpenAI

Settings.llm = OpenAI(
    model="gpt-4o-mini",
    api_base="https://api.n4n.ai/v1",
    api_key="YOUR_KEY",
)

Now define a tool and an OpenAI agent.

from llama_index.core.tools import FunctionTool
from llama_index.agents.openai import OpenAIAgent

def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b

tool = FunctionTool.from_defaults(fn=add)
agent = OpenAIAgent.from_tools([tool], verbose=True)

The agent will call the model, possibly invoke add, and loop until it returns a final answer. Each of those frames becomes a span with proper parent-child links.

Step 5: Run the agent and emit traces

Execute a query.

response = agent.chat("What is 17 plus 25?")
print(response)

With the console exporter, you will see span objects printed as JSON. Look for span names like chat, llm, function_tool. Each span carries attributes: model name, token usage, tool input/output.

If you shipped to OTLP, open your tracing UI and find the trace by operation name OpenAIAgent.chat. You should see a parent span for the agent turn, child spans for the LLM completion, and a child span for the add tool. The LLM span typically includes llm.model and llm.token_count.total (attribute names vary slightly by LlamaIndex version).

Step 6: Verify success

A correct setup satisfies three checks:

1. Spans exist. You see at least one span with llama_index instrumentation scope in your exporter output.
2. Hierarchy is correct. The agent span is parent to the LLM span and the tool span, not flattened.
3. Attributes are populated. The LLM span includes model and token counts.

Run this minimal assertion against the console output or query your collector’s API:

from opentelemetry.trace import get_tracer_provider
from opentelemetry.sdk.trace import TracerProvider

assert isinstance(get_tracer_provider(), TracerProvider)

For programmatic verification in tests, use an in-memory span exporter:

from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter

exporter = InMemorySpanExporter()
provider.add_span_processor(BatchSpanProcessor(exporter))

# ... run agent ...

spans = exporter.get_finished_spans()
assert any("agent" in span.name.lower() for span in spans)
assert any(span.attributes.get("llm.model") for span in spans)

If those assertions pass, your tracing LlamaIndex agents with OpenTelemetry pipeline is working end to end.

Understanding the span tree

A typical agent turn produces this shape:

• OpenAIAgent.chat (parent)
  • llm (model completion, may repeat if the agent loops)
  • function_tool (one per tool call)
    • any custom spans you create inside the tool

The parent span duration equals wall-clock time for the whole turn. Child LLM spans reveal retries: if you see two llm spans with the same prompt, the first likely hit a rate limit or produced invalid JSON. Tool spans show serialization overhead—often the hidden cost in agents that call many small functions.

Reducing noise with span processors

Agents emit a lot of spans. In high-throughput services, sample or filter. A custom processor can drop spans from health-check tools:

from opentelemetry.sdk.trace import SpanProcessor

class DropHealthToolProcessor(SpanProcessor):
    def on_start(self, span, parent_context):
        if span.name == "function_tool" and "health" in str(span.attributes):
            span.set_attribute("otel.outcome", "SUPPRESSED")

    def on_end(self, span):
        pass

    def shutdown(self):
        pass

provider.add_span_processor(DropHealthToolProcessor())

Keep the built-in BatchSpanProcessor for export; add custom processors before it.

Testing in CI

Add a smoke test that runs a dummy agent against a fake LLM and asserts span emission. Use InMemorySpanExporter and llama_index.llms.mock if available, or monkeypatch the chat method. This catches regressions where someone moves the OTel init after LlamaIndex import.

def test_agent_emits_spans():
    exporter = InMemorySpanExporter()
    provider.add_span_processor(BatchSpanProcessor(exporter))
    # build agent with mock LLM, run one chat
    spans = exporter.get_finished_spans()
    assert len(spans) > 0

What to do when spans go missing

If you see no spans, check these:

• Import order. The OTel tracer provider and LlamaIndex span handler must be set before LlamaIndex imports its instrumentation. Put Step 2 and Step 3 at the top of your entrypoint.
• Async loops. If you use arun, the same spans apply, but BatchSpanProcessor flushes on process exit. Call provider.force_flush() before your script ends.
• Disabled instrumentation. Some LlamaIndex versions gate instrumentation behind a setting. Ensure it is not turned off.

Adding custom attributes to agent spans

The built-in handler covers the framework, but your business logic needs context. Use the OTel API inside a tool to add attributes.

from opentelemetry import trace

def add(a: int, b: int) -> int:
    span = trace.get_current_span()
    span.set_attribute("app.user_id", "12345")
    span.set_attribute("app.tenant", "acme")
    return a + b

These attributes show up on the tool span, letting you filter traces by user or tenant in your backend.

Closing notes on cost and routing

Tracing exposes token counts per step, which is the only real way to debug agent cost. If you route through a gateway that honors client routing directives and forwards provider cache-control hints, those hints appear in span attributes when the instrumentation includes them. That visibility lets you confirm whether a prompt prefix actually hit the provider’s cache.

Tracing LlamaIndex agents with OpenTelemetry is not optional once you move past toy scripts. The spans tell you which tool blew the context window, which model call retried, and where latency accumulated.