Large Flow Language Models

One-line takeaway

Lafa is a continuous flow language model scaled to 8B parameters: blockwise flow with causal attention on the prefix and bidirectional asynchronous flow within blocks, trained by adapting pretrained autoregressive models and served fast via prefix caching and blockwise parallel decoding, at 0.5B, 7B and 8B scales.

Lafa 是扩展到 8B 参数的连续流语言模型：块内双向异步流、前缀因果注意力，可以从预训练自回归模型改造来训练，推理时用前缀缓存和块并行解码提速；训练了 0.5B、7B 和 8B 三个规模。