ECHO: Efficient Causal Self-Recursion Beyond Auxiliary Multi-Token Prediction

One-line takeaway

ECHO has the backbone's own final layer predict every future horizon instead of auxiliary MTP modules, with Dynamic Depth Sampling keeping cost down; pretraining gains concentrate on reasoning and code at half the overhead of training every depth.

ECHO 让主干网络最后一层直接预测多个未来 token，取代外挂模块式的多 token 预测，并用动态深度采样控制训练成本；预训练中收益集中在推理和代码，训练开销是全深度 MTP 的一半。