Topic
Serving Framework Benchmarks
13 posts on serving framework benchmarks — part of benchmarks & performance on the n4n AI blog.
Why batch size changes the vLLM vs TGI latency picture
Batch size reshapes the vLLM vs TGI latency tradeoff: at low concurrency TGI edges out, but vLLM's paged attention scales better under heavy load. Here's why.
SGLang vs vLLM: latency benchmark for structured outputs
A head-to-head latency and throughput comparison of SGLang and vLLM for structured JSON outputs, with benchmarks, ergonomics, and verdict.
Ollama vs vLLM: latency benchmark for local deployment
A head-to-head benchmark of Ollama and vLLM for local LLM serving: latency, throughput, cost, and ergonomics to help you choose the right stack.
How speculative decoding compares across serving frameworks
A head-to-head speculative decoding serving frameworks comparison of vLLM and TGI across capabilities, latency, cost, ergonomics, and limits for engineers.
How PagedAttention changed LLM serving throughput
PagedAttention rewired LLM serving to boost throughput via paged KV caches. This analysis covers how vLLM uses it, tradeoffs, and when to adopt it.
Benchmarking vLLM vs LMDeploy for high-concurrency serving
Practical head-to-head of vLLM and LMDeploy for high-concurrency LLM serving: capabilities, throughput, ergonomics, limits, and a clear verdict for engineers.
Benchmarking SGLang's RadixAttention for multi-turn chat
A practical analysis of SGLang's RadixAttention for multi-turn chat workloads, with real tradeoffs, code patterns, and a decisive recommendation.
vLLM vs TGI vs SGLang: throughput benchmark compared
A practical engineer's guide to the vllm vs tgi vs sglang benchmark tradeoffs across throughput, latency, ergonomics, and cost models for self-hosting LLMs.
TGI vs vLLM: which serves Llama 3 70B faster?
A pragmatic head-to-head of TGI vs vLLM for serving Llama 3 70B: latency, throughput, cost, ergonomics, and which to deploy for your production workload.
Choosing a serving framework: vLLM, TGI, or SGLang in 2026
A practical guide to choosing vllm tgi sglang 2026: match serving frameworks to your workload, benchmark correctly, and avoid deployment pitfalls.
Benchmarking vLLM's continuous batching against TGI
A practical head-to-head look at vllm continuous batching vs tgi across scheduling, throughput, cost, ergonomics, and limits, with a verdict for self-hosting LLMs.
Benchmarking TensorRT-LLM vs vLLM on an H100 cluster
A practitioner's head-to-head tensorrt-llm vs vllm h100 benchmark across throughput, cost, ergonomics, and limits, with a clear framework selection verdict.
Benchmarking KV cache efficiency: vLLM vs SGLang vs TGI
Practical comparison of kv cache efficiency vllm sglang tgi across capabilities, cost, latency, and ergonomics to help engineers pick a serving framework.
More topics in benchmarks & performance
- Agentic Workflow Performance Benchmarks14
- Benchmark Methodology and Measurement14
- Code Generation Latency for Dev Tools14
- Flagship Model Speed Showdown14
- Llama 4 Inference Speed by Provider14
- Price-Performance Rankings14
- Provider Uptime and Reliability Benchmarks14
- Reasoning Model Latency Overhead14
- Customer Support Chatbot Latency13
- DeepSeek Performance Benchmarks13
- GPU Inference Benchmarks13
- Long-Context Latency Benchmarks13