// ⚡ Inference Acceleration & Serving

KV Cache

Stores past keys and values so each decoding step doesn't recompute them.

Foundational core

Learning resources ↓ What to learn first →

IN PLAIN WORDS

Each new token attends to all previous ones; caching their K/V means only the new token is computed each step. Memory grows linearly with context — the problem GQA, MLA and PagedAttention address.

Used in: Every attention-based LLM inference engine; linear-attention models (e.g. Laplace 3B) replace it with a fixed-size state

TIMELINE · prerequisites → this → what builds on it

2017 Transformer &…2019 Multi-Query A…2022 Speculative D…2022 Continuous (I…2023 PagedAttentio…2023 Attention Sin…

01Original paper

! No single verifiable original paper

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Transformer & Self-Attention

KV Cache

Then you can learn

Related:Grouped-Query Attention (GQA)

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Official docs

KV Cache ↗EN

Explainers

Reference code

rasbt/LLMs-from-scratch ↗EN

04Where the course teaches it

U60 Decoding, Sampling & KV Cache Prefill and decode, temperature/top-p, KV cache and the memory-bandwidth bottleneck. 6 h4 algorithms2 paperslesson ready U63 Running LLMs Locally llama.cpp, Ollama, MLX, memory estimates and choosing a model. 4 h3 algorithms0 papers

On learning paths:Understanding an LLM End to End

For AI / data

Verification: pending, checked 2026-10-22.
JSON: /api/learn/algorithms.json (id: kv-cache)