// ✳ Mixture of Experts
DeepSeekMoE (Fine-Grained & Shared Experts)
Finer-grained experts plus always-on shared experts for specialization.
DeepSeekMoE 细粒度专家 2024 Advanced core
IN PLAIN WORDS
Split experts into many smaller ones and activate more of them for flexible combinations, plus shared experts every token uses for common knowledge.
Used in: DeepSeek-V2/V3/R1, Qwen MoE models
TIMELINE · prerequisites → this → what builds on it
01Original paper
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Damai Dai, Chengqi Deng, Chenggang Zhao, et al. · 2024
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Then you can learn
Related:Multi-head Latent Attention (MLA)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
On learning paths:Understanding an LLM End to End
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
deepseek-moe)