// ∿ New Architectures: SSM & Linear Attention
Gated Delta Networks
Linear attention combining delta-rule memory updates with gated forgetting.
门控 DeltaNet 2024 Research frontier core
IN PLAIN WORDS
Linear attention accumulates into a memory matrix that saturates. The delta rule overwrites old values before writing and gating handles forgetting; together they beat Mamba-2 on recall and long context.
Used in: Hybrid linear-attention models such as Qwen3-Next; PyTorch TorchTitan integrated Gated DeltaNet kernels for training (Oct 2026); KronLSTM (2610.26834) benchmarks against it
TIMELINE · prerequisites → this → what builds on it
01Original paper
Gated Delta Networks: Improving Mamba2 with Delta Rule
Songlin Yang, Jan Kautz, Ali Hatamizadeh · 2024
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Then you can learn
- Nothing in the atlas builds on it yet
Related:Hybrid Architectures (Jamba)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
Reference code
04Where the course teaches it
On learning paths:Generative Models & Frontier Research
For AI / data
- Verification: verified, checked 2026-10-22.
- JSON: /api/learn/algorithms.json (id:
gated-deltanet)