// ◈ Quantization, Pruning & Distillation

LLM.int8() & Outlier Handling

8-bit matmul with outlier features kept in 16-bit.

LLM.int8() 与离群值 2022 Intermediate core

IN PLAIN WORDS

LLM activations contain a few huge outlier dimensions that break naive quantization. LLM.int8() keeps them in FP16 and quantizes the rest to INT8, halving memory almost losslessly.

Used in: bitsandbytes, Hugging Face 8-bit loading

TIMELINE · prerequisites → this → what builds on it

2017 Mixed Precisi…2022 LLM.int8() & Outl…2022 GPTQ2022 SmoothQuant2023 QLoRA2024 BitNet b1.58 …

01Original paper

LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

Tim Dettmers, Mike Lewis, Younes Belkada, et al. · 2022

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Mixed Precision & FP8 Training

LLM.int8() & Outlier Handling

Then you can learn

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Official docs

04Where the course teaches it

U61 Quantization, Pruning & Distillation INT8/INT4, GPTQ, AWQ, SmoothQuant, BitNet and knowledge distillation. 8 h7 algorithms4 papers U63 Running LLMs Locally llama.cpp, Ollama, MLX, memory estimates and choosing a model. 4 h3 algorithms0 papers

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: quantization-int8)