Topic

Mixture of Experts (MoE): DeepSeek, Mixtral & Grok

7 posts on mixture of experts (moe): deepseek, mixtral & grok — part of glossary on the n4n AI blog.

What is mixture of experts? MoE explained simply

Mixture of Experts (MoE) explained for engineers: how sparse routing cuts compute, real architecture details, and what gets misunderstood.

July 28, 20264 min read

GlossaryComparison

MoE vs dense models: what's the tradeoff?

A practitioner's comparison of MoE vs dense models across training cost, inference latency, memory needs, and model quality — with a verdict by use case.

July 28, 20266 min read

GlossaryAnalysis

How Mixtral 8x7B uses mixture of experts

Deep dive into Mixtral 8x7B's mixture-of-experts architecture, routing mechanics, and practical tradeoffs for engineers deploying sparse models.

July 28, 20265 min read

GlossaryAnalysis

How many experts activate per token in MoE models?

How many experts activate per token in MoE models like Mixtral, DeepSeek, and Grok, and what it means for inference cost, latency, and model quality.

July 28, 20268 min read

GlossaryAnalysis

How does Grok-1 use a mixture-of-experts design?

Technical deep-dive into Grok-1's MoE architecture: 314B params, 8 experts, top-2 routing, and what it means for inference engineering.

July 28, 20265 min read

GlossaryComparison

DeepSeek-V3 vs Mixtral: comparing two MoE architectures

Technical comparison of DeepSeek-V3 and Mixtral MoE architectures across routing, inference economics, ecosystem, and real-world deployment trade-offs.

July 28, 20267 min read

GlossaryAnalysis

DeepSeek-V3's mixture-of-experts architecture explained

DeepSeek-V3's MoE architecture breaks down: 671B params, 37B active, multi-head latent attention, and the routing tradeoffs engineers actually face.

July 28, 20266 min read