// ✳ Mixture of Experts
Sparsely-Gated Mixture of Experts
A gating network routes each input to only a few expert sub-networks.
稀疏门控混合专家 2017 Intermediate core
IN PLAIN WORDS
Split a layer into many experts; a router picks the top-k for each token. Total parameters can be huge while per-token compute stays small.
Used in: Origin of modern MoE LLMs
TIMELINE · prerequisites → this → what builds on it
01Original paper
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, et al. · 2017
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Related:DeepSeekMoE (Fine-Grained & Shared Experts)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
On learning paths:Understanding an LLM End to End
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
sparse-moe)