// ▲ Pretraining, Scaling & Distributed Training
Chinchilla Compute-Optimal Scaling
At fixed compute, scale parameters and tokens together (~20 tokens/param).
Chinchilla 计算最优 2022 Intermediate core
IN PLAIN WORDS
DeepMind found prior LLMs over-sized and under-trained; 70B Chinchilla on 1.4T tokens beat 280B Gopher. Many models now overtrain beyond this ratio to cut inference cost.
Used in: Data budgeting for LLM training
01Original paper
Training Compute-Optimal Large Language Models
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al. · 2022
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Then you can learn
- Nothing in the atlas builds on it yet
Related:μP (Maximal Update Parametrization)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
On learning paths:Understanding an LLM End to End
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
chinchilla)