// ⚡ Inference Acceleration & Serving
Speculative Decoding
A small draft model proposes tokens that the big model verifies in one pass.
推测解码 2022 Intermediate core
IN PLAIN WORDS
Decoding is memory-bound with idle compute. A draft model proposes k tokens and the target verifies them in parallel with a rejection rule that preserves the exact output distribution, giving 2–3× speedups.
Used in: vLLM, SGLang, TensorRT-LLM, llama.cpp; DLite (arXiv 2610.23787, Oct 2026) drafts without history for ever-growing agent contexts
TIMELINE · prerequisites → this → what builds on it
01Original paper
Fast Inference from Transformers via Speculative Decoding
Yaniv Leviathan, Matan Kalman, Yossi Matias · 2022
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Related:Multi-Token Prediction (MTP)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
Explainers
Official docs
04Where the course teaches it
On learning paths:AI Application Engineer
For AI / data
- Verification: verified, checked 2026-10-22.
- JSON: /api/learn/algorithms.json (id:
speculative-decoding)