// ∿ New Architectures: SSM & Linear Attention

xLSTM

Upgrades LSTM with exponential gating and matrix memory.

2024 Advanced extended

IN PLAIN WORDS

LSTM's originators propose sLSTM (exponential gating) and mLSTM (parallelizable matrix memory), stacking them into LLMs competitive with Transformers and Mamba.

Used in: xLSTM 7B

TIMELINE · prerequisites → this → what builds on it

1997 Long Short-Te…2024 xLSTM

01Original paper

xLSTM: Extended Long Short-Term Memory

Maximilian Beck, Korbinian Pöppel, Markus Spanring, et al. · 2024

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Long Short-Term Memory (LSTM)

xLSTM

Then you can learn

Nothing in the atlas builds on it yet

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Reference code

github.com ↗EN

04Where the course teaches it

U43 MoE & Beyond-Transformer Architectures Sparse experts, load balancing, Mamba/SSMs, linear attention and hybrid architectures, diffusion language models. 10 h18 algorithms5 papers

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: xlstm)