// ◎ Transformers & Attention
YaRN (Context Window Extension)
Rescales RoPE frequencies to extend context with little fine-tuning.
YaRN 上下文扩展 2023 Advanced core
IN PLAIN WORDS
Models break beyond trained length. YaRN interpolates RoPE dimensions differently by frequency and rescales attention temperature, extending 4K to 64K–128K with brief fine-tuning.
Used in: Long-context Qwen and DeepSeek models
01Original paper
YaRN: Efficient Context Window Extension of Large Language Models
Bowen Peng, Jeffrey Quesnelle, Honglu Fan, et al. · 2023
arXiv ↗ ✓ Verified · arXiv API
02Where it sits
Learn the left column first; the right column builds on this one.
Learn first
Then you can learn
- Nothing in the atlas builds on it yet
Related:Ring Attention
Left: prerequisites · right: builds on this · dashed: related. Full map →
03Learning resources
Official docs, explainers, courses and reference code. All links checked.
Reference code
Explainers
04Where the course teaches it
For AI / data
- Verification: verified, checked 2026-10-21.
- JSON: /api/learn/algorithms.json (id:
yarn)