Topic

RLHF, DPO & Instruction Tuning

7 posts on rlhf, dpo & instruction tuning — part of glossary on the n4n AI blog.

What is RLHF? Reinforcement learning from human feedback

A technical explainer of RLHF — how reward modeling, PPO, and human preference data align LLMs to follow instructions and avoid harmful outputs.

July 28, 20267 min read

GlossaryDefinition

What is instruction tuning? A practical explanation

A practical guide to instruction tuning — what it is, how it differs from pretraining and RLHF, and why it matters for building usable LLMs.

July 28, 20267 min read

GlossaryDefinition

What is DPO? Direct preference optimization explained

Direct Preference Optimization (DPO) explained for engineers — how it works, why it replaces PPO in RLHF, a concrete training example, and common misconceptions.

July 28, 20267 min read

GlossaryComparison

RLHF vs DPO: how the two alignment methods differ

A practical comparison of RLHF and DPO alignment methods covering training dynamics, compute costs, implementation complexity, and when to use each approach.

July 28, 202610 min read

GlossaryHow-to

How RLHF trains a reward model

A step-by-step guide to training a reward model for RLHF, from preference data preparation through validation and deployment for PPO.

July 28, 20264 min read

GlossaryAnalysis

How ChatGPT was trained with RLHF

A technical breakdown of ChatGPT's RLHF pipeline — supervised fine-tuning, reward modeling, and PPO — with code sketches and honest tradeoffs.

July 28, 20266 min read

GlossaryComparison

Constitutional AI vs RLHF: how Claude is aligned

Technical comparison of Constitutional AI and RLHF alignment methods, covering training pipelines, trade-offs, and when to use each approach.

July 28, 20265 min read