Deep Fusion Memory: A Chunk-Level Parametric Memory for Large Language Models

One-line takeaway

Deep Fusion Memory is a chunk-level parametric memory for LLMs: a causal memory predictor generates memory from preceding context and injects it into intermediate hidden states rather than interpolating outputs, avoiding continued-pretraining forgetting and inference-time retrieval cost while cutting offline data-construction work.

Deep Fusion Memory 是给大模型的块级参数化记忆：用因果记忆预测器根据前文生成记忆，并把学到的记忆注入中间隐藏层，而不是只在输出层插值；目标是避开持续预训练的遗忘和推理时检索的开销，并减少构造记忆训练数据所需的离线检索。