// ▲ Pretraining, Scaling & Distributed Training
Scaling Laws
Loss falls as a smooth power law in parameters, data and compute.
缩放定律 2020 Foundational core
IN PLAIN WORDS
Loss follows power laws in model size, data and compute over many orders of magnitude, so small runs predict big ones and guide compute allocation.
Used in: Training plans of every frontier lab
TIMELINE · prerequisites → this → what builds on it
01Original paper
Scaling Laws for Neural Language Models
Jared Kaplan, Sam McCandlish, Tom Henighan, et al. · 2020
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Then you can learn
Related:Test-Time Compute Scaling
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
On learning paths:Understanding an LLM End to End
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
scaling-laws)