// ⌖ Interpretability

Superposition

Models pack more features than dimensions by superimposing them.

叠加假说 2022 Advanced core

IN PLAIN WORDS

Toy models show that sparse features get stored in nearly orthogonal directions, making neurons polysemantic. This explains why neuron-level reading fails and motivated sparse autoencoders.

Used in: Theory underpinning interpretability

TIMELINE · prerequisites → this → what builds on it

1901 Principal Com…2022 Superposition2023 Sparse Autoen…

01Original paper

Toy Models of Superposition

Nelson Elhage, Tristan Hume, Catherine Olsson, et al. · 2022

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Principal Component Analysis (PCA)

Superposition

Then you can learn

Sparse Autoencoders (SAE)

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Explainers

Transformer Circuits Thread ↗EN

04Where the course teaches it

U91 Interpretability: from Attribution to Circuits Attribution, probes, the logit lens, superposition, SAEs, circuit tracing, knowledge editing and activation steering. 10 h9 algorithms4 paperslesson ready

On learning paths:Generative Models & Frontier Research

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: superposition)