图像编辑与可控生成 · CVPR2026 ·2026.06.03

Dynamic-eDiTor:基于多模态扩散 Transformer 的免训练文本驱动 4D 场景编辑

Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformer

Dong In Lee、Hyungjun Doh、Seunggeun Chi、Runlin Duan、Sangpil Kim、Karthik Ramani

与你有关的部分

把 MM-DiT 的注意力改造成时空子网格注意力并配合词元继承与光流引导的词元替换,可在不训练的情况下把 2D 编辑能力迁移到多视角视频,直接优化已有 4DGS;这套「免训练 + 词元传播」思路可复用到任何需要跨视角/跨帧一致性的扩散编辑任务。

原文依据 Spatio-Temporal Sub-Grid Attention (STGA) for locally consistent cross-view and temporal fusion, and Context Token Propagation (CTP)
可迁移到 文本驱动视频编辑多视角一致性生成动态场景重建

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

近期 4D 表示(如动态神经辐射场与 4D 高斯泼溅(4DGS))的进展使动态 4D 场景重建成为可能。然而,文本驱动的 4D 场景编辑仍研究不足,难点在于编辑过程中需同时保证跨视角与跨时间的一致性。现有工作依赖独立编辑各帧的 2D 扩散模型,常导致运动畸变、几何漂移与编辑不完整。我们提出 Dynamic-eDiTor,一个免训练的文本驱动 4D 编辑框架,基于多模态扩散 Transformer(MM-DiT)与 4DGS。该机制包含用于局部一致的跨视角与时序融合的时空子网格注意力(STGA),以及通过词元继承与光流引导的词元替换实现全局传播的上下文词元传播(CTP)。二者结合使 Dynamic-eDiTor 无需额外训练即可生成无缝、全局一致的多视角视频,并直接优化预训练的源 4DGS。在多视角视频数据集 DyNeRF 上的大量实验表明,本方法在编辑保真度以及多视角与时序一致性上均优于先前方法。结果与代码见项目主页:https://di-lee.github.io/dynamic-eDiTor

英文原文摘要 1288 字符

Recent progress in 4D representations, such as Dynamic NeRF and 4D Gaussian Splatting (4DGS), has enabled dynamic 4D scene reconstruction. However, text-driven 4D scene editing remains under-explored due to the challenge of ensuring both multi-view and temporal consistency across space and time during editing.Existing studies rely on 2D diffusion models that edit frames independently, often causing motion distortion, geometric drift, and incomplete editing. We introduce Dynamic-eDiTor, a training-free text-driven 4D editing framework leveraging Multimodal Diffusion Transformer (MM-DiT) and 4DGS. This mechanism consists of Spatio-Temporal Sub-Grid Attention (STGA) for locally consistent cross-view and temporal fusion, and Context Token Propagation (CTP) for global propagation via token inheritance and optical-flow-guided token replacement. Together, these components allow Dynamic-eDiTor to perform seamless, globally consistent multi-view video without additional training and directly optimize pre-trained source 4DGS.Extensive experiments on multi-view video dataset DyNeRF demonstrate that our method achieves superior editing fidelity and both multi-view and temporal consistency prior approaches. Project page for results and code: https://di-lee.github.io/dynamic-eDiTor

标签

会议 CVPR2026
页码 1187-1197
Bibkey
来源 CVF Open Access