// ∿ New Architectures: SSM & Linear Attention

Hyena Hierarchy

Replaces attention with implicitly parameterized long convolutions and gating.

Hyena 长卷积 2023 Research frontier extended

IN PLAIN WORDS

A small network generates sequence-length filters computed via FFT, mixed with multiplicative gating, at near-linear cost. Strong on very long sequences like DNA.

Used in: Genomic models (e.g., HyenaDNA, Evo)

TIMELINE · prerequisites → this → what builds on it

2021 Structured St…2023 Hyena Hierarchy

01Original paper

Hyena Hierarchy: Towards Larger Convolutional Language Models

Michael Poli, Stefano Massaroli, Eric Nguyen, et al. · 2023

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Structured State Spaces (S4)

Hyena Hierarchy

Then you can learn

Nothing in the atlas builds on it yet

Related:Mamba (Selective State Spaces)

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Reference code

04Where the course teaches it

U43 MoE & Beyond-Transformer Architectures Sparse experts, load balancing, Mamba/SSMs, linear attention and hybrid architectures, diffusion language models. 10 h18 algorithms5 papers

For AI / data

Verification: verified, checked 2026-10-22.
JSON: /api/learn/algorithms.json (id: hyena)