GenHOI:面向对象一致的手物交互,采用时序均衡与空间选择性的对象注入
GenHOI: Towards Object-Consistent Hand-Object Interaction with Temporally Balanced and Spatially Selective Object Injection
Head-Sliding RoPE 通过给参考 token 分配头级时序偏移来缓解 3D RoPE 的时序衰减,这一技巧可直接迁移到任何以参考图做长视频主体一致性的生成/编辑模型;两级空间注意力门控则提供了「只在交互区域增强条件、保留背景真实感」的可复用条件注入方案。
原文依据 we propose Head-Sliding RoPE, which assigns head-specific temporal offsets to reference tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论手物交互(HOI)仍是数字人视频合成中的核心挑战,模型必须生成物理上合理的接触并在各帧间保持对象身份一致。尽管近期的 HOI 重演方法取得了进展,但它们通常在域内训练与评测,难以泛化到复杂的真实场景。相比之下,一体化视频编辑模型表现出更广的鲁棒性,却仍在对象外观不一致等 HOI 特有问题上表现不佳。本文提出 GenHOI,一种对预训练视频生成模型的轻量级增强,以时序均衡且空间选择的方式注入参考对象信息。在时序均衡方面,我们提出 Head-Sliding RoPE,为参考 token 分配头级时序偏移,使其影响在各帧间均匀分布,并缓解 3D RoPE 的时序衰减,从而提升长程对象一致性。在空间选择性方面,我们设计了两级空间注意力门控,将对象条件注意力集中于 HOI 区域并自适应缩放其强度,在增强交互保真度的同时保持背景真实感。在未见过的真实场景上的大量定性与定量评测表明,GenHOI 显著优于最优的 HOI 重演与一体化视频编辑对比方法。
英文原文摘要 1417 字符
Hand-Object Interaction (HOI) remains a core challenge in digital human video synthesis, where models must generate physically plausible contact and preserve object identity across frames. Although recent HOI reenactment approaches have achieved progress, they are typically trained and evaluated in-domain and fail to generalize to complex, in-the-wild scenarios. In contrast, all-in-one video editing models exhibit broader robustness but still struggle with HOI-specific issues such as inconsistent object appearance. In this paper, we present GenHOI, a lightweight augmentation to pretrained video generation models that injects reference-object information in a temporally balanced and spatially selective manner. For temporal balancing, we propose Head-Sliding RoPE, which assigns head-specific temporal offsets to reference tokens, distributing their influence evenly across frames and mitigating the temporal decay of 3D RoPE to improve long-range object consistency. For spatial selectivity, we design a two-level spatial attention gate that concentrates object-conditioned attention on HOI regions and adaptively scales its strength, preserving background realism while enhancing interaction fidelity. Extensive qualitative and quantitative evaluations on unseen, in-the-wild scenes demonstrate that GenHOI significantly outperforms state-of-the-art HOI reenactment and all-in-one video editing competitors.