// ◎ Transformers & Attention
Native Sparse Attention (NSA)
Trainable, hardware-aligned dynamic sparse attention.
原生稀疏注意力 NSA 2025 Research frontier core
IN PLAIN WORDS
Combines three branches — compressed coarse blocks, selected top blocks and a sliding window — merged by gates. Sparse from pretraining, it speeds both long-context training and inference.
Used in: DeepSeek long-context research; the same team later shipped a different fine-grained sparse attention (DSA) in DeepSeek-V3.2-Exp
TIMELINE · prerequisites → this → what builds on it
01Original paper
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Jingyang Yuan, Huazuo Gao, Damai Dai, et al. · 2025
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Then you can learn
- Nothing in the atlas builds on it yet
Related:Multi-head Latent Attention (MLA)
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
Reference code
04Where the course teaches it
On learning paths:Generative Models & Frontier Research
For AI / data
- Verification: verified, checked 2026-10-22.
- JSON: /api/learn/algorithms.json (id:
nsa)