Object-WIPER：免训练的视频物体及其关联效果移除
Object-WIPER: Training-Free Object and Associated Effect Removal in Videos
利用预训练视频扩散 Transformer 的交叉注意力与自注意力定位目标物体及其效果，并通过反转-去噪流程在免训练条件下完成视频物体移除与修复，为视频编辑提供可复用的注意力引导与背景保持策略。
原文依据 we localize relevant visual tokens via visual-text cross-attention and visual self-attention
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 Object-WIPER，一个免训练框架，用于从视频中移除动态物体及其关联的视觉效果，并用语义一致且时序连贯的内容进行修复。该方法利用预训练的文本到视频扩散 Transformer（DiT）。给定输入视频、用户提供的物体掩码以及描述目标物体及其效果的查询词元，我们通过视觉-文本交叉注意力和视觉自注意力定位相关的视觉词元。由此生成一个中间效果掩码，并将其与用户掩码融合，得到最终需要替换的前景词元掩码。我们首先通过 DiT 对视频进行反转以获得结构化噪声，然后用高斯噪声重新初始化被掩码的词元，同时保留背景词元。在去噪过程中，我们复制反转期间保存的背景词元值以保持场景保真度。为解决缺乏合适评估的问题，我们引入一个新的物体移除指标，该指标奖励连续帧间前景词元的时序一致性、每帧内前景与背景词元之间的连贯性，以及输入与输出前景词元之间的差异性。在 DAVIS 和新构建的真实世界关联效果基准 WIPER-Bench 上的实验表明，Object-WIPER 在该指标上超越了基于训练和免训练的基线方法，在无需任何重新训练的情况下实现了干净的移除和时序稳定的重建。我们的新基准、源代码和预训练模型将公开可用。
英文原文摘要 1584 字符
In this paper, we introduce Object-WIPER, a training-free framework for removing dynamic objects and their associated visual effects from videos, and inpainting them with semantically consistent and temporally coherent content. Our approach leverages a pre-trained text-to-video diffusion transformer (DiT). Given an input video, a user-provided object mask, and query tokens describing the target object and its effects, we localize relevant visual tokens via visual-text cross-attention and visual self-attention. This produces an intermediate effect mask that we fuse with the user mask to obtain a final foreground token mask to replace. We first invert the video through the DiT to obtain structured noise, then reinitialize the masked tokens with Gaussian noise while preserving background tokens. During denoising, we copy values for the background tokens saved during inversion to maintain scene fidelity. To address the lack of suitable evaluation, we introduce a new object removal metric that rewards temporal consistency among foreground tokens across consecutive frames, coherence between foreground and background tokens within each frame, and dissimilarity between the input and output foreground tokens. Experiments on DAVIS and a newly curated real-world associated effect benchmark WIPER-Bench show that Object-WIPER surpasses both training-based and training-free baselines in terms of the metric, achieving clean removal and temporally stable reconstruction without any retraining. Our new benchmark, source code, and pre-trained models will be publicly available.