基于结构与接触感知表征的开放世界手物交互视频生成
Open-world Hand-Object Interaction Video Generation Based on Structure and Contact-aware Representation
提出一种无需3D标注的结构与接触感知表征（手物接触、遮挡、整体结构上下文），可作为辅助监督信号嵌入视频生成模型，并用共享-特化联合生成范式同时生成表征与视频；对做可控/物理真实视频生成的人，这是一个可复用的表征监督与联合训练思路。
原文依据 we propose a structure and contact-aware representation that captures hand-object contact, hand-object occlusion, and holistic structure context without 3D annotations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论生成逼真的手物交互（HOI）视频是一项重大挑战，原因在于难以建模物理约束（例如手与被操作物体之间的接触与遮挡）。现有方法将 HOI 表征作为辅助生成目标来引导视频合成。然而，2D 与 3D 表征之间存在两难：无法同时保证可扩展性与交互保真度。为解决这一局限，我们提出一种结构与接触感知表征，无需 3D 标注即可捕捉手物接触、手物遮挡以及整体结构上下文。这种面向交互且可扩展的监督信号使模型能够学习细粒度的交互物理规律，并泛化到开放世界场景。为充分利用所提出的表征，我们引入一种联合生成范式，采用共享与特化策略同时生成面向交互的表征与视频。大量实验表明，我们的方法在两个真实世界数据集上生成物理真实且时序连贯的 HOI 视频方面优于最优方法。此外，我们的方法在具有挑战性的开放世界场景中展现出强泛化能力，凸显了可扩展设计的优势。
英文原文摘要 1324 字符
Generating realistic hand-object interactions (HOI) videos is a significant challenge due to the difficulty of modeling physical constraints (e.g., contact and occlusion between hands and manipulated objects). Current methods utilize HOI representation as an auxiliary generative objective to guide video synthesis. However, there is a dilemma between 2D and 3D representations that cannot simultaneously guarantee scalability and interaction fidelity. To address this limitation, we propose a structure and contact-aware representation that captures hand-object contact, hand-object occlusion, and holistic structure context without 3D annotations. This interaction-oriented and scalable supervision signal enables the model to learn fine-grained interaction physics and generalize to open-world scenarios. To fully exploit the proposed representation, we introduce a joint-generation paradigm with a share-and-specialization strategy that generates interaction-oriented representations and videos. Extensive experiments demonstrate that our method outperforms state-of-the-art methods on two real-world datasets in generating physics-realistic and temporally coherent HOI videos. Furthermore, our approach exhibits strong generalization to challenging open-world scenarios, highlighting the benefit of our scalable design.