Topic
Structured Logging for LLM APIs
13 posts on structured logging for llm apis — part of developer tools on the n4n AI blog.
What to log for every OpenAI-compatible chat completion call
A precise definition of what to log for chat completion calls on OpenAI-compatible APIs, with structured logging fields, code examples, and pitfalls.
Structured logs vs raw text logs for debugging LLM errors
Structured logs vs raw text logs for LLM debugging: a head-to-head comparison across capabilities, cost, latency, ergonomics, and ecosystem for engineers.
Structured logging vs full tracing: when each one wins
A practical head-to-head on structured logging vs tracing for LLM APIs across cost, latency, ergonomics, and limits, with a detailed engineering verdict per use case.
Structured logging fields every LLM API call should capture
Essential structured logging fields for LLM APIs: trace IDs, token usage, latency, routing, and cost metrics to observably run model calls.
Shipping LLM logs to Elasticsearch with structured fields
Learn how to ship LLM logs to Elasticsearch with structured fields in this hands-on tutorial covering schema design, Python instrumentation, and bulk indexing.
Redacting sensitive data in LLM request/response logs
Step-by-step guide to redacting sensitive data in LLM logs: learn how to intercept, classify, mask, and verify PII in request and response payloads safely.
Logging prompts and completions without leaking PII
Practical steps to implement logging LLM prompts without leaking PII: redaction, hashing, structured schemas, and verification for production LLM apps.
Log sampling strategies for high-volume LLM traffic
Practical log sampling strategies for high-volume LLM traffic: how to retain signal, cut costs, and debug failures without drowning in data.
Log LLM latency, tokens, and cost in a single event schema
Learn how to design a unified LLM logging schema that captures latency, token usage, and cost in one event, with code, queries, and pitfalls for engineers.
JSON logging for LLM requests: a practical schema
A practical JSON logging schema for LLM requests: design a field-by-field structured log with code examples and pitfalls for engineers building LLM apps.
Designing a log schema for multi-provider LLM gateways
Practical guide to designing a log schema for multi-provider LLM gateways: capture request IDs, attempts, tokens, routing, and failures without bloat.
Correlating request IDs across LLM retries and fallbacks
Learn how to implement correlating request IDs across LLM retries and fallbacks with structured logging, OpenAI-compatible clients, and runnable Python code.
Building a structured logger for streaming LLM responses
Learn to build a structured logger for streaming responses from LLM APIs. Step-by-step Python tutorial with runnable code and sample JSON output.
More topics in developer tools
- CI/CD Pipelines for LLM Apps15
- LLM Evaluation Frameworks15
- Cost Optimization & Model Routing14
- Debugging Hallucinations & Output Quality14
- LangChain Debugging & Observability14
- LLM Observability Platforms14
- Migrating Between LLM Providers14
- Model Deprecation & Version Migration14
- RAG Pipeline Observability14
- Regression Testing for Prompts14
- Compliance & Audit Logging for Regulated Industries13
- CrewAI & AutoGen Multi-Agent Debugging13