Topic
Latency, Throughput & Time-to-First-Token
6 posts on latency, throughput & time-to-first-token — part of glossary on the n4n AI blog.
What is tokens per second in LLM benchmarking?
Tokens per second measures LLM output throughput. Learn how it's calculated, why it differs from latency, and what it means for real-time applications.
What is time-to-first-token in LLM inference?
Time-to-first-token (TTFT) measures the latency from request send to first generated token — critical for streaming UX and system design.
Latency vs throughput: what's the difference for LLMs?
Understand the latency vs throughput trade-off in LLM inference — what each metric measures, how they conflict, and which to optimize for your use case.
How to measure LLM API latency correctly
A practical guide to measuring LLM API latency with correct methodology, covering time-to-first-token, inter-token latency, and end-to-end metrics with runnable code.
How prompt length affects time-to-first-token
How prompt length directly impacts time-to-first-token in LLM inference, with concrete analysis of prefill costs, attention mechanics, and system design tradeoffs.
Batch size vs latency: the inference trade-off
Understand the batch size vs latency trade-off in LLM inference — how batching affects throughput, memory, and time-to-first-token across real serving scenarios.
More topics in glossary
- Structured Outputs & JSON Mode19
- AI Agents Fundamentals12
- Hallucination in LLMs11
- Sampling Parameters: Top-p, Top-k & Penalties11
- Context Window & Context Length10
- Fine-Tuning Fundamentals9
- Foundation Models: Base vs Instruct vs Chat9
- Model Families & Naming Conventions: GPT-5, Claude, Gemini 3, Llama 4, Mistral, DeepSeek, Qwen, Grok9
- Grounding & Fact-Checking in AI8
- LLM Benchmarks: MMLU, HumanEval, SWE-bench & GPQA8
- Max Tokens, Stop Sequences & Output Truncation8
- Quantization Formats: GGUF, GPTQ, AWQ & INT4/INT88