// ✳ Mixture of Experts
Mixtral (Top-2 MoE)
An open MoE that picks 2 of 8 experts per token.
Mixtral（top-2 MoE） 2024 Intermediate core
IN PLAIN WORDS
Each layer has 8 FFN experts and uses 2 per token — ~47B total, ~13B active — matching larger dense models. It popularized open MoE.
Used in: Mixtral 8x7B / 8x22B
TIMELINE · prerequisites → this → what builds on it
01Original paper
Mixtral of Experts
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, et al. · 2024
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Mixtral (Top-2 MoE)
Then you can learn
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
mixtral)