// ◈ Quantization, Pruning & Distillation

GPTQ

Second-order, column-by-column post-training weight quantization to 3–4 bits.

GPTQ 训练后量化 2022 Advanced core

IN PLAIN WORDS

After quantizing each column, compensate the error on remaining columns using an approximate Hessian. With little calibration data it quantizes 175B models in hours on one GPU.

Used in: Many GPTQ-format open weights

TIMELINE · prerequisites → this → what builds on it

2022 LLM.int8() & …2022 GPTQ2023 AWQ (Activati…2023 SparseGPT

01Original paper

GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

Elias Frantar, Saleh Ashkboos, Torsten Hoefler, et al. · 2022

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

LLM.int8() & Outlier Handling

GPTQ

Then you can learn

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Reference code

ModelCloud/GPTQModel ↗EN

Official docs

huggingface.co ↗EN

04Where the course teaches it

U61 Quantization, Pruning & Distillation INT8/INT4, GPTQ, AWQ, SmoothQuant, BitNet and knowledge distillation. 8 h7 algorithms4 papers U63 Running LLMs Locally llama.cpp, Ollama, MLX, memory estimates and choosing a model. 4 h3 algorithms0 papers

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: gptq)