Topic
Self-Attention & Multi-Head Attention
4 posts on self-attention & multi-head attention — part of glossary on the n4n AI blog.
GlossaryDefinition
What is self-attention? The mechanism explained simply
Self-attention explained for engineers: how query/key/value projections weight token relationships, why multi-head attention matters, and common misconceptions.
6 min read
GlossaryDefinition
What is multi-head attention and why use multiple heads?
A precise technical explanation of multi-head attention in transformers, covering mechanics, purpose, and common misconceptions for engineers building LLM systems.
5 min read
GlossaryTutorial
Self-attention mechanism explained with a worked example
A hands-on tutorial implementing the self-attention mechanism from scratch with NumPy, including a complete worked example with numerical outputs.
3 min read
GlossaryTutorial
Query, key, and value vectors in attention explained
A hands-on tutorial implementing query, key, and value attention from scratch with NumPy, showing how self-attention computes relationships between tokens.
5 min read
More topics in glossary
- Structured Outputs & JSON Mode19
- AI Agents Fundamentals12
- Hallucination in LLMs11
- Sampling Parameters: Top-p, Top-k & Penalties11
- Context Window & Context Length10
- Fine-Tuning Fundamentals9
- Foundation Models: Base vs Instruct vs Chat9
- Model Families & Naming Conventions: GPT-5, Claude, Gemini 3, Llama 4, Mistral, DeepSeek, Qwen, Grok9
- Grounding & Fact-Checking in AI8
- LLM Benchmarks: MMLU, HumanEval, SWE-bench & GPQA8
- Max Tokens, Stop Sequences & Output Truncation8
- Quantization Formats: GGUF, GPTQ, AWQ & INT4/INT88