Topic
Quantization Fundamentals
5 posts on quantization fundamentals — part of glossary on the n4n AI blog.
What is quantization in AI? A plain-English guide
A practical guide to quantization in AI — what it is, how it works, why engineers use it, and the trade-offs you'll actually face in production.
What is 4-bit quantization and why does it matter?
A precise technical definition of 4-bit quantization, how it compresses LLM weights, practical trade-offs, and what engineers get wrong about accuracy loss.
How model quantization reduces memory and cost
A practical guide to quantizing LLMs with step-by-step code for GGUF, GPTQ, and AWQ formats, plus verification methods and deployment trade-offs.
How many parameters can you run on 24GB of VRAM?
Calculate exactly how many model parameters fit in 24GB VRAM across quantization levels, with formulas, overhead breakdowns, and real-world deployment tradeoffs.
Does quantization hurt model accuracy?
Quantization trades precision for speed and memory — here's when accuracy drops, when it doesn't, and how to pick the right scheme for your workload.
More topics in glossary
- Structured Outputs & JSON Mode19
- AI Agents Fundamentals12
- Hallucination in LLMs11
- Sampling Parameters: Top-p, Top-k & Penalties11
- Context Window & Context Length10
- Fine-Tuning Fundamentals9
- Foundation Models: Base vs Instruct vs Chat9
- Model Families & Naming Conventions: GPT-5, Claude, Gemini 3, Llama 4, Mistral, DeepSeek, Qwen, Grok9
- Grounding & Fact-Checking in AI8
- LLM Benchmarks: MMLU, HumanEval, SWE-bench & GPQA8
- Max Tokens, Stop Sequences & Output Truncation8
- Quantization Formats: GGUF, GPTQ, AWQ & INT4/INT88