音频同步视频实例编辑与粒度感知掩码精修器
Audio-sync Video Instance Editing with Granularity-Aware Mask Refiner
把用户给出的粗掩码迭代精修为实例级精确区域，并引入音频代理提供细粒度时序控制，为视频编辑中的空间-时间可控性提供了可复用的掩码精修与音频条件注入思路。
原文依据 a granularity-aware mask refiner that iteratively refines coarse user-provided masks into precise instance-level regions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期视频生成领域的进展表明，逼真的音视频同步对于富有吸引力的内容创作至关重要。然而，现有的视频编辑方法在很大程度上忽视了音视频同步，并且缺乏精确的实例级编辑所需的细粒度空间与时序可控性。本文提出 AVI-Edit，一个用于音频同步视频实例编辑的框架。我们提出一种粒度感知掩码精修器，将用户提供的粗掩码迭代精修为精确的实例级区域。我们进一步设计了一个自反馈音频代理来筛选高质量的音频引导，从而提供细粒度的时序控制。为促进该任务，我们还构建了一个大规模数据集，具有以实例为中心的对应关系和全面的标注。大量实验表明，AVI-Edit 在视觉质量、条件遵循和音视频同步方面优于最优方法。项目主页：https://hjzheng.net/projects/AVI-Edit/。
英文原文摘要 1027 字符
Recent advancements in video generation highlight that realistic audio-visual synchronization is crucial for engaging content creation. However, existing video editing methods largely overlook audio-visual synchronization and lack the fine-grained spatial and temporal controllability required for precise instance-level edits. In this paper, we propose AVI-Edit, a framework for audio-sync video instance editing. We propose a granularity-aware mask refiner that iteratively refines coarse user-provided masks into precise instance-level regions. We further design a self-feedback audio agent to curate high-quality audio guidance, providing fine-grained temporal control. To facilitate this task, we additionally construct a large-scale dataset with instance-centric correspondence and comprehensive annotations. Extensive experiments demonstrate that AVI-Edit outperforms state-of-the-art methods in visual quality, condition following, and audio-visual synchronization. Project page: https://hjzheng.net/projects/AVI-Edit/.