Topic
KV Cache
6 posts on kv cache — part of glossary on the n4n AI blog.
What is PagedAttention and how does it manage KV cache?
PagedAttention explained: how vLLM pages KV cache like virtual memory, eliminating fragmentation and enabling efficient batching for LLM inference.
What is a KV cache in transformer inference?
A precise technical explanation of KV cache in transformer inference — how it works, why it matters for latency, and what engineers get wrong about it.
How long context windows strain the KV cache
Long context windows explode KV cache memory. Here's why it happens, what it costs, and the tradeoffs engineers actually face when scaling past 32K tokens.
How KV cache quantization reduces GPU memory
Learn how KV cache quantization cuts GPU memory usage during LLM inference with practical quantization strategies and verification steps.
How KV cache memory usage scales with context length
Understand how KV cache memory scales with context length, including formulas, real-world tradeoffs, and optimization strategies for LLM inference.
How grouped-query attention shrinks the KV cache
How grouped-query attention reduces KV cache memory by sharing keys and values across query heads, with concrete math and tradeoff analysis.
More topics in glossary
- Structured Outputs & JSON Mode19
- AI Agents Fundamentals12
- Hallucination in LLMs11
- Sampling Parameters: Top-p, Top-k & Penalties11
- Context Window & Context Length10
- Fine-Tuning Fundamentals9
- Foundation Models: Base vs Instruct vs Chat9
- Model Families & Naming Conventions: GPT-5, Claude, Gemini 3, Llama 4, Mistral, DeepSeek, Qwen, Grok9
- Grounding & Fact-Checking in AI8
- LLM Benchmarks: MMLU, HumanEval, SWE-bench & GPQA8
- Max Tokens, Stop Sequences & Output Truncation8
- Quantization Formats: GGUF, GPTQ, AWQ & INT4/INT88