// ⚡ Inference Acceleration & Serving

Attention Sinks (StreamingLLM)

Keeping a few initial sink tokens enables endless streaming generation.

注意力汇聚点 StreamingLLM 2023 Intermediate extended

IN PLAIN WORDS

Models dump attention on the first few tokens; dropping them in a sliding window breaks generation. Keeping them plus a recent window streams millions of tokens stably.

Used in: Long streaming conversations

TIMELINE · prerequisites → this → what builds on it

2020 Sliding-Windo…2023 Attention Sinks (…

01Original paper

Efficient Streaming Language Models with Attention Sinks

Guangxuan Xiao, Yuandong Tian, Beidi Chen, et al. · 2023

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Attention Sinks (StreamingLLM)

Then you can learn

Nothing in the atlas builds on it yet

Related:Sparse Autoencoders (SAE)

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Reference code

github.com ↗EN

04Where the course teaches it

U60 Decoding, Sampling & KV Cache Prefill and decode, temperature/top-p, KV cache and the memory-bandwidth bottleneck. 6 h4 algorithms2 paperslesson ready

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: streaming-llm)