// ⚡ Algorithms Atlas
Inference Acceleration & Serving
KV cache, speculative decoding, PagedAttention, continuous batching — making LLM serving fast and cheap.
6 core4 extended2019–2024
TIMELINE
01Core algorithms
Learn these first. Sorted by year.
Temperature & Nucleus (Top-p) Sampling Controls randomness by sampling only from the top-p probability mass. 2019core Speculative Decoding A small draft model proposes tokens that the big model verifies in one pass. 2022core Continuous (Iteration-Level) Batching Re-forms the batch every decoding step so finished requests leave immediately. 2022core PagedAttention (vLLM) Manages KV cache in pages like OS virtual memory. 2023core EAGLE Speculates at the feature level for higher acceptance rates. 2024core KV Cache Stores past keys and values so each decoding step doesn't recompute them. core
02Extended
Variants and follow-ups worth knowing once the core is clear.
RadixAttention (Prefix Caching) Reuses KV cache across requests sharing prefixes via a radix tree. 2023ext Attention Sinks (StreamingLLM) Keeping a few initial sink tokens enables endless streaming generation. 2023ext Medusa Adds extra decoding heads so the model drafts its own speculative tokens. 2024ext Prefill–Decode Disaggregation Runs prompt prefill and token decoding on separate GPU pools. 2024ext
03Course units that teach these
U02 Hallucination, Reliability & Limits Why models confidently make things up, and how to check and reduce the risk. 2 h2 algorithms1 papers U60 Decoding, Sampling & KV Cache Prefill and decode, temperature/top-p, KV cache and the memory-bandwidth bottleneck. 6 h4 algorithms2 paperslesson ready U62 Serving Engines PagedAttention, continuous batching, prefix caching, speculative decoding, prefill/decode disaggregation and gateway routing. 8 h7 algorithms4 papers U63 Running LLMs Locally llama.cpp, Ollama, MLX, memory estimates and choosing a model. 4 h3 algorithms0 papers U83 AI Engineering: Evals, Cost & Shipping Your own eval sets, LLM judges, observability, model gateways, cost control and product design. 8 h3 algorithms1 papers