ECHO: Efficient Causal Self-Recursion Beyond Auxiliary Multi-Token Prediction

一句话要点

ECHO 让主干网络最后一层直接预测多个未来 token，取代外挂模块式的多 token 预测，并用动态深度采样控制训练成本；预训练中收益集中在推理和代码，训练开销是全深度 MTP 的一半。

ECHO has the backbone's own final layer predict every future horizon instead of auxiliary MTP modules, with Dynamic Depth Sampling keeping cost down; pretraining gains concentrate on reasoning and code at half the overhead of training every depth.