// ∿ New Architectures: SSM & Linear Attention

Looped Language Models

Re-applies the same layers several times, trading depth in parameters for depth in compute.

循环语言模型（Looped LM） aka Universal Transformer / Recurrent-depth Transformer / 循环深度 2018 Advanced extended

IN PLAIN WORDS

Intuition: instead of stacking 100 distinct layers, run one block over the representation several times, each pass adding a step of thought. Parameters stay fixed while extra loops buy more reasoning at inference. Universal Transformers introduced it; Ouro (2025) scaled it to pre-training; every extra loop costs training and decoding compute.

Used in: Ouro looped LMs (ByteDance Seed et al.); Oct 2026 work on looped decision models (LEV), fixed-point training and progress-aligned recurrence (PAR, 2610.27628)

TIMELINE · prerequisites → this → what builds on it

2017 Transformer &…2018 Looped Language M…

01Original paper

Universal Transformers

Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, et al. · 2018

arXiv ↗ ✓ Verified · DataCite (arXiv DOI 10.48550/arXiv.1807.03819)

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Transformer & Self-Attention

Looped Language Models

Then you can learn

Nothing in the atlas builds on it yet

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Official docs

ouro-llm.github.io ↗EN

Reference code

04Where the course teaches it

U43 MoE & Beyond-Transformer Architectures Sparse experts, load balancing, Mamba/SSMs, linear attention and hybrid architectures, diffusion language models. 10 h18 algorithms5 papers

For AI / data

Verification: verified, checked 2026-10-22.
JSON: /api/learn/algorithms.json (id: looped-lm)