// ∿ New Architectures: SSM & Linear Attention

Structured State Spaces (S4)

Models very long sequences efficiently with structured state-space equations.

结构化状态空间 S4 2021 Research frontier extended

IN PLAIN WORDS

Treat the sequence as input to a linear dynamical system; a special matrix structure lets it run as an RNN at inference and as a convolution in training.

Used in: Long sequences, audio, genomics

TIMELINE · prerequisites → this → what builds on it

1997 Long Short-Te…2021 Structured State …2023 Mamba (Select…2023 Hyena Hierarc…

01Original paper

Efficiently Modeling Long Sequences with Structured State Spaces

Albert Gu, Karan Goel, Christopher Ré · 2021

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Long Short-Term Memory (LSTM)

Structured State Spaces (S4)

Then you can learn

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Explainers

The Annotated S4 ↗EN

04Where the course teaches it

U43 MoE & Beyond-Transformer Architectures Sparse experts, load balancing, Mamba/SSMs, linear attention and hybrid architectures, diffusion language models. 10 h18 algorithms5 papers

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: s4)