// ⬡ Deep Learning Foundations
GELU
A smooth activation that gates inputs by their Gaussian probability.
GELU 激活函数 2016 Foundational core
IN PLAIN WORDS
Multiplies the input by the standard normal CDF, a smoother ReLU. Used in BERT and GPT-family models.
Used in: BERT, GPT-2/3, ViT
01Original paper
Gaussian Error Linear Units (GELUs)
Dan Hendrycks, Kevin Gimpel · 2016
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
For AI / data
- Verification: verified, checked 2026-10-22.
- JSON: /api/learn/algorithms.json (id:
gelu)