以图导图：基于整流上下文示例引导的一致视频非模态补全
Image Guides Images: Consistent Video Amodal Completion with Rectified In-Context Exemplar Guidance
把预训练图像修复模型直接用于视频非模态补全，通过调制 DiT 全局注意力中示例与目标帧的相对权重来提升跨帧一致性，并用双帧校准的示例修正解决冷启动；这套免训练、示例引导的注意力整流思路可迁移到其他需要跨帧一致性的视频生成/编辑任务。
原文依据 we propose a training-free framework by utilizing pre-trained image inpainting models for VAC and introducing in-context (IC) learning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频非模态补全（VAC）旨在模拟人脑隐式感知被部分遮挡物体完整外观的能力，从而促进识别与理解。现有 VAC 方法在自建数据集上微调视频生成模型，但由于采集真实非模态数据困难，这些数据集往往分布不真实且规模较小，因而限制了其性能与泛化能力。为此，我们提出一种免训练框架，利用预训练图像修复模型完成 VAC，并引入上下文（IC）学习以增强帧间一致性。然而，尽管基于 DiT 的上下文学习在生成任务中表现令人满意，任务无关的全局注意力倾向于利用无关的上下文信息，导致应用于非模态补全时出现补全失败。此外，上下文学习在示例构建上面临冷启动问题。为此，我们提出基于整流上下文示例引导的一致视频非模态补全。具体而言，我们引入整流示例引导补全，通过调整示例相对于目标图像的注意力权重来实现一致补全，并采用双帧校准的示例整流以解决冷启动问题。定量与定性实验表明，我们的方法优于当前最优方法，尤其在非常见数据以及严重遮挡条件下的泛化性与鲁棒性方面表现突出。代码已开源：https://github.com/xxyCSCV/IC-Amodal。
英文原文摘要 1579 字符
Video amodal completion (VAC) aims to mimic the human brain's ability to implicitly perceive the complete appearance of partially occluded objects, thereby facilitating recognition and understanding. Existing VAC methods finetune video generation models on custom datasets, yet these datasets often have unrealistic distributions and small scales due to the challenges of collecting real amodal data, and thus limit their performance and generalization. To address this, we propose a training-free framework by utilizing pre-trained image inpainting models for VAC and introducing in-context (IC) learning to enhance inter-frame consistency. However, despite the satisfactory performance of DiT-based IC Learning in generation tasks, task-agnostic global attention tends to utilize irrelevant contextual information, resulting in completion failures when applied to amodal completion. Additionally, IC Learning faces a cold-start problem with the exemplar construction. To this end, we propose a consistent video amodal completion with rectified in-context exemplar guidance. Specifically, we introduce rectified exemplar-guided completion by adjusting the attention weights of exemplar relative to the target images for consistent completion, and adopt a dual-frame calibrated exemplar rectification to tackle the cold-start issue. Quantitative and qualitative experiments demonstrate that our method outperforms SOTAs, especially in terms of generalization and robustness on uncommon data and under severe occlusion. Codes are available at https://github.com/xxyCSCV/IC-Amodal.