// ⬡ Deep Learning Foundations
Weight Initialization (Xavier / He)
Scale initial weights by layer width so signals and gradients neither explode nor vanish.
权重初始化（Xavier / He） aka Kaiming Init / Glorot Init / He 初始化 2010 Foundational core
IN PLAIN WORDS
Weights that start too large blow signals up layer after layer; too small and they fade to zero, so deep nets will not train. Xavier init sets the variance from fan-in and fan-out (good for tanh); He init doubles it for ReLU. Framework defaults today descend from these two ideas.
Used in: Default layer initializers in PyTorch, JAX and other frameworks
01Original paper
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot, Yoshua Bengio · AISTATS 2010 (PMLR v9) · 2010
Open ↗ ✓ Verified · 官方页面 HTTP 核实
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Then you can learn
- Nothing in the atlas builds on it yet
Related:Batch NormalizationResidual Networks (ResNet)μP (Maximal Update Parametrization)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
For AI / data
- Verification: verified, checked 2026-10-22.
- JSON: /api/learn/algorithms.json (id:
weight-init)