// ⬡ Deep Learning Foundations
AdamW (Decoupled Weight Decay)
Decouples weight decay from Adam's gradient update.
AdamW 2017 Foundational core
IN PLAIN WORDS
Adding L2 to Adam's gradient gets distorted by adaptive scaling; AdamW shrinks weights directly. It is the standard optimizer for LLM training.
Used in: Pretraining of GPT, Llama and most LLMs
TIMELINE · prerequisites → this → what builds on it
01Original paper
Decoupled Weight Decay Regularization
Ilya Loshchilov, Frank Hutter · 2017
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
AdamW (Decoupled Weight Decay)
Then you can learn
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
On learning paths:Zero to Deep Learning
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
adamw)