// ⬡ Deep Learning Foundations
Layer Normalization
Normalizes within each example instead of across the batch.
层归一化 2016 Foundational core
IN PLAIN WORDS
Normalizes across a token's features, so it works with variable-length sequences and small batches. The Transformer default.
Used in: BERT, GPT-2 and other Transformers
TIMELINE · prerequisites → this → what builds on it
01Original paper
Layer Normalization
Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton · 2016
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Layer Normalization
Then you can learn
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
U31 Training Deep Nets SGD/momentum/Adam/AdamW/Muon, learning-rate schedules, BatchNorm/LayerNorm/RMSNorm, dropout and residuals. 10 h10 algorithms4 papers U41 Transformers & Attention Self-attention, multi-head attention, positional encoding, residuals and normalisation — build a GPT from scratch. 16 h6 algorithms3 paperslesson ready
On learning paths:Zero to Deep Learning
For AI / data
- Verification: verified, checked 2026-10-22.
- JSON: /api/learn/algorithms.json (id:
layernorm)