// ∿ Algorithms Atlas
New Architectures: SSM & Linear Attention
Post-Transformer directions: Mamba, RWKV, linear attention, hybrids and diffusion language models.
5 core6 extended2018–2025
TIMELINE
01Core algorithms
Learn these first. Sorted by year.
Mamba (Selective State Spaces) Input-dependent state spaces with linear-time cost and Transformer-like quality. 2023core RWKV Trains in parallel like a Transformer and infers like an RNN. 2023core Gated Delta Networks Linear attention combining delta-rule memory updates with gated forgetting. 2024core Hybrid Architectures (Jamba) Interleaves attention and Mamba layers (plus MoE) in one model. 2024core Diffusion Language Models (LLaDA) Generates text by iteratively unmasking a fully masked sequence. 2025core
02Extended
Variants and follow-ups worth knowing once the core is clear.
Looped Language Models Re-applies the same layers several times, trading depth in parameters for depth in compute. 2018ext Structured State Spaces (S4) Models very long sequences efficiently with structured state-space equations. 2021ext Retentive Network (RetNet) An attention alternative with parallel, recurrent and chunkwise modes. 2023ext Hyena Hierarchy Replaces attention with implicitly parameterized long convolutions and gating. 2023ext Mamba-2 (State Space Duality) Shows SSMs and a class of attention are dual, yielding faster Mamba. 2024ext xLSTM Upgrades LSTM with exponential gating and matrix memory. 2024ext