arXiv 2610.13630 · cs.CL · 2026-10-09
Multi-Stream Attention Residuals
Xiaohan Qin, Xiangdong Zhang, Tuo Dai, Sunan Zou, Huaijin Wu, Yebin Yang, Zhuo Xia, Lin Yao and 2 more
Shanghai Jiao Tong University
In latest · #230 Added on 2026-10-21
One-line takeaway
Multi-Stream Attention Residuals extend depth-wise attention residuals to multiple heads, with sizeable gains at little extra cost.
多流注意力残差把“沿网络深度的注意力”扩展为多头形式，以很小开销带来明显提升。
Read on arXiv PDF Hugging Face discussion Code
Topics & disciplines
More on LLMs
- Jeva: System One Models from First Principles ▲371
- Post-Training Leaves Behavioral Shadows on Unrelated Decisions ▲273
- OmniEdu: Open Foundation Models for Learning and Teaching ▲240
- PAR: Progress-Aligned Recurrence for Length Generalization in State Tracking ▲148
- MemT-1.0: Towards Memory-Native Evolution ▲119
About this data
Metadata from the arXiv API (verified); upvotes from Hugging Face Daily Papers; citations from Semantic Scholar. Last refreshed 2026-10-22.
The takeaway is written by AIMap from the abstract; the abstract itself is not reproduced — read the original.