Distributed tracing falls apart the moment your LLM call leaves the synchronous request path. Proper OpenTelemetry context propagation async code requires explicit handling of context across asyncio tasks, background workers, and callback chains, or your spans silently orphan and latency attribution becomes guesswork. This guide walks through a concrete Python asyncio + httpx + OpenTelemetry setup that keeps traces continuous from inbound HTTP request to streaming LLM response.
Step 1: Install and initialize the OpenTelemetry SDK
Start with the OTLP exporter and a tracer provider. Skip the all-in-one auto-instrumentation packages if you need precise control over async boundaries—they often miss cross-task spans.
pip install opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-api httpxInitialize once at process start, before any request handling:
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
resource = Resource.create({"service.name": "llm-orchestrator"})
provider = TracerProvider(resource=resource)
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("llm-orchestrator")The tracer is globally available, but the active span lives in a contextvars slot managed by the opentelemetry.context module. That slot does not automatically follow your code into every concurrency primitive.
Step 2: Propagate context across asyncio tasks
On Python 3.11+, asyncio.create_task copies contextvars into the child task. On earlier versions, or when you use loop.call_soon, run_in_executor, or asyncio.to_thread, the OTel context is lost. Capture and re-attach explicitly.
from opentelemetry import context as otel_context
import asyncio
async def traced_task(coro):
ctx = otel_context.get_current()
def wrapper():
token = otel_context.attach(ctx)
try:
return asyncio.run(coro)
finally:
otel_context.detach(token)
return await asyncio.to_thread(wrapper)For pure asyncio, use the built-in context wrapper:
async def child_span():
with tracer.start_as_current_span("child"):
await asyncio.sleep(0.1)
async def parent():
with tracer.start_as_current_span("parent"):
ctx = otel_context.get_current()
# with_current_context attaches ctx inside the coroutine when it runs
task = asyncio.create_task(otel_context.with_current_context(child_span())())
await taskIf you omit with_current_context on Python <3.11, the child span becomes a root span with a new trace ID. That is the most common bug in OpenTelemetry context propagation async pipelines.
Step 3: Inject trace headers into async LLM calls
When you POST to an OpenAI-compatible endpoint, the HTTP client must emit traceparent and tracestate headers. Use propagation.inject on a mutable headers dict.
from opentelemetry.propagate import inject
import httpx
async def call_llm(prompt: str):
headers = {}
inject(headers) # adds traceparent/tracestate
async with httpx.AsyncClient(timeout=30.0) as client:
resp = await client.post(
"https://api.openai.com/v1/chat/completions",
headers=headers,
json={"model": "gpt-4o", "messages": [{"role": "user", "content": prompt}]},
)
resp.raise_for_status()
return resp.json()If you route through a gateway such as n4n.ai, the same injection works against its single OpenAI-compatible endpoint; the gateway honors client routing directives and forwards provider cache-control hints, but your client retains the parent span. Wrap the call in a span to capture latency and token counts:
async def traced_llm_call(prompt):
with tracer.start_as_current_span("llm.call") as span:
span.set_attribute("llm.prompt", prompt[:200])
result = await call_llm(prompt)
usage = result.get("usage", {})
span.set_attribute("llm.total_tokens", usage.get("total_tokens", 0))
return resultWithout the inject step, the LLM provider’s internal spans (if it emits any) cannot nest under your trace, and you lose the ability to correlate provider-side latency with your orchestration logic.
Step 4: Extract context in downstream workers
If your LLM fan-out pushes jobs to a queue (Redis, SQS, Celery), HTTP header injection is useless. Serialize the OTel context into the job payload and extract it on the consumer.
from opentelemetry.propagate import inject, extract
import json, redis
r = redis.Redis()
def enqueue_job(job_data: dict):
carrier = {}
inject(carrier)
job_data["otel_carrier"] = carrier
r.rpush("llm_jobs", json.dumps(job_data))
def process_job(raw: str):
job = json.loads(raw)
ctx = extract(job.get("otel_carrier", {}))
token = otel_context.attach(ctx)
try:
with tracer.start_as_current_span("worker.process"):
# call LLM or run post-processing
...
finally:
otel_context.detach(token)This restores the parent trace ID so worker spans nest under the original request. Forgetting to extract produces a flat trace with disconnected worker nodes—a classic symptom of broken OpenTelemetry context propagation async handoffs.
Step 5: Bind context to streaming callbacks
LLM streaming and tool-call callbacks fire in separate coroutine chains. Capture the context at registration, not at call time.
def make_callback(original_ctx):
async def callback(chunk: dict):
token = otel_context.attach(original_ctx)
try:
with tracer.start_as_current_span("llm.stream_chunk"):
# e.g., accumulate or forward chunk
pass
finally:
otel_context.detach(token)
return callback
async def stream_llm(prompt):
ctx = otel_context.get_current()
cb = make_callback(ctx)
# hypothetical async generator from an LLM SDK
async for chunk in fake_stream(prompt):
await cb(chunk)If you register the callback without binding original_ctx, the streaming spans appear under a fresh root and your trace breaks exactly where latency matters most: during token generation.
Step 6: Verify end-to-end trace continuity
Point the exporter at a collector (OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317) and run a request that triggers at least two async LLM calls plus one queued worker. Inspect the trace in Jaeger or Tempo.
Success criteria:
- The root span (
POST /chat) has child spansllm.callfor each async task, with correctparent_id. - Worker spans show the same
trace_idas the root and a non-nullparent_id. - No orphan spans with
parent_idempty that correspond to known LLM paths.
A lightweight unit test using the in-memory exporter:
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
def test_propagation():
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)
t = trace.get_tracer("test")
with t.start_as_current_span("root") as root:
ctx = otel_context.get_current()
with otel_context.with_current_context(t.start_as_current_span("child"))() as child:
pass
spans = exporter.get_finished_spans()
assert spans[1].parent == root.get_span_context()Gotchas that will bite you
asyncio.to_threaddoes not copy OTel context. Attach manually as shown in Step 2.- FastAPI
BackgroundTasksrun after the response is sent; the request span is closed unless you pass the captured context into the task. - If your root span is sampled out (e.g.,
TraceIdRatioBasedat low rate), all children vanish. UseParentBased(AlwaysOn)in development. - Context attach returns a token; failing to detach leaks contextvars and can cause spans to cross wires under load.
OpenTelemetry context propagation async patterns add a few lines per boundary, but they are the difference between a trace that explains a slow multi-model fan-out and a dashboard of unexplained latency. Implement the inject/extract pair at every I/O edge and your LLM observability will survive any concurrency trick you throw at it.