// ✳ Mixture of Experts
Auxiliary-Loss-Free Load Balancing
Balances expert load with dynamic biases instead of an extra loss.
无辅助损失负载均衡 2024 Advanced core
IN PLAIN WORDS
Balancing losses interfere with the main objective. Instead, add a per-expert bias to routing scores, lowered when overloaded and raised when idle.
Used in: DeepSeek-V3
01Original paper
Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
Lean Wang, Huazuo Gao, Chenggang Zhao, et al. · 2024
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Auxiliary-Loss-Free Load Balancing
Then you can learn
- Nothing in the atlas builds on it yet
Related:Switch Transformer
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
Reference code
04Where the course teaches it
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
aux-free-balancing)