让你的图像随你的运动而动!——隐式多目标多运动迁移
Let Your Image Move with Your Motion! -- Implicit Multi-Object Multi-Motion Transfer
把扩散注意力本身当作物体掩码的来源,用物体特定掩码约束注意力,使运动 token 与文本 token 只作用于指定区域,这一「注意力即掩码」的思路可直接迁移到多主体视频编辑、组合式可控生成等需要区域解耦的任务。
原文依据 we introduce a Motion Decoupled Mask Attention Mechanism that uses object-specific masks to constrain attention
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论运动迁移已成为可控视频生成中一个有前景的方向,但现有方法大多聚焦于单目标场景,当多个目标需要各自不同的运动模式时便难以应对。在本工作中,我们提出 FlexiMMT,这是首个显式支持多目标、多运动迁移的隐式图像到视频(I2V)运动迁移框架。给定一张静态多目标图像和多段参考视频,FlexiMMT 独立地提取运动表示并将其准确地分配给不同目标,支持灵活的重组以及任意的运动到目标映射。为应对跨目标运动纠缠这一核心挑战,我们引入了运动解耦掩码注意力机制,该机制利用目标特定的掩码来约束注意力,确保运动 token 与文本 token 只影响其指定区域。我们进一步提出了差异化掩码传播机制,该机制直接从扩散注意力中导出目标特定的掩码,并在帧间高效地逐步传播。大量实验表明,FlexiMMT 在基于 I2V 的多目标多运动迁移中实现了精确、可组合且最优的性能。我们的项目主页为:https://ethan- li123.github.io/FlexiMMT_page/
英文原文摘要 1306 字符
Motion transfer has emerged as a promising direction for controllable video generation, yet existing methods largely focus on single-object scenarios and struggle when multiple objects require distinct motion patterns. In this work, we present FlexiMMT, the first implicit image-to-video (I2V) motion transfer framework that explicitly enables multi-object, multi-motion transfer. Given a static multi-object image and multiple reference videos, FlexiMMT independently extracts motion representations and accurately assigns them to different objects, supporting flexible recombination and arbitrary motion-to-object mappings. To address the core challenge of cross-object motion entanglement, we introduce a Motion Decoupled Mask Attention Mechanism that uses object-specific masks to constrain attention, ensuring that motion and text tokens only influence their designated regions. We further propose a Differentiated Mask Propagation Mechanism that derives object-specific masks directly from diffusion attention and progressively propagates them across frames efficiently. Extensive experiments demonstrate that FlexiMMT achieves precise, compositional, and state-of-the-art performance in I2V-based multi-object multi-motion transfer. Our project page is: https://ethan- li123.github.io/FlexiMMT_page/