OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing

One-line takeaway

Adobe Research's OmniXRef is a unified multimodal multi-reference video editing framework that combines variable numbers of text instructions, image references and video references; a reference-bank interface with R^2-RoPE generalizes to more references, and a single-stream diffusion transformer processes source and references together.

Adobe Research 的 OmniXRef 是多模态、多参考的视频编辑框架：可以组合任意数量的文字指令、图像参考（如人物外观）和视频参考（如镜头运动）来改原视频；用参考库接口和 R^2-RoPE 支持更多参考，模型是单流扩散 Transformer。