// ▲ Pretraining, Scaling & Distributed Training
Mixed Precision & FP8 Training
Compute in 16- or 8-bit floats while keeping FP32 master weights.
混合精度与 FP8 训练 2017 Intermediate core
IN PLAIN WORDS
Low-precision math is faster and lighter; FP32 master weights and loss scaling prevent degradation. FP8 doubles throughput again but needs fine-grained scaling.
Used in: Nearly all LLM training; DeepSeek-V3 uses FP8
TIMELINE · prerequisites → this → what builds on it
01Original paper
Mixed Precision Training
Paulius Micikevicius, Sharan Narang, Jonah Alben, et al. · 2017
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Then you can learn
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
04Where the course teaches it
On learning paths:Understanding an LLM End to End
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
mixed-precision)