Topic

LLM-as-a-Judge & Model Evaluation

6 posts on llm-as-a-judge & model evaluation — part of glossary on the n4n AI blog.

What is LLM-as-a-judge? A practical introduction

A practical definition of LLM-as-a-judge with code examples, evaluation patterns, and common pitfalls engineers encounter when automating model assessment.

July 28, 20265 min read

GlossaryComparison

LLM-as-a-judge vs human evaluation: pros and cons

A practitioner's comparison of LLM-as-a-judge and human evaluation across cost, latency, reliability, and operational trade-offs for production LLM systems.

July 28, 20267 min read

GlossaryHow-to

How to evaluate a RAG pipeline with an LLM judge

A step-by-step guide to building an LLM-as-a-judge evaluator for RAG pipelines, with runnable code for retrieval quality, answer faithfulness, and end-to-end correctness.

July 28, 20265 min read

GlossaryHow-to

How to build an LLM-as-a-judge pipeline

A step-by-step guide to building a production-grade LLM-as-a-judge evaluation pipeline with code examples, calibration techniques, and verification strategies.

July 28, 20264 min read

GlossaryDefinition

How MT-Bench evaluates multi-turn conversations

MT-Bench explained: how the LLM-as-a-judge benchmark evaluates multi-turn conversation quality with 80 questions across 8 categories.

July 28, 20265 min read

GlossaryAnalysis

Common failure modes of LLM-as-a-judge systems

A practitioner's breakdown of the systematic biases that make LLM judges unreliable — positional bias, verbosity preference, sycophancy, calibration drift, and context leakage — with concrete mitigation patterns.

July 28, 20264 min read