Interact2Ar:基于自回归扩散模型的全身体人-人交互生成
Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Models
面向人体交互运动生成,提出并行分支建模手部运动学与带记忆机制的自回归扩散框架,可直接复用于手部精细动作生成、长时序运动合成与实时扰动自适应等视觉-运动生成任务。
原文依据 the first end-to-end text-conditioned autoregressive diffusion model for generating full-body, human-human interactions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论生成逼真的人-人交互是一项具有挑战性的任务,不仅需要高质量的个体身体与手部运动,还需要所有交互参与者之间协调一致。受限于可用数据与更高的学习复杂度,以往方法往往忽略手部运动,从而限制了交互的真实感与表现力。此外,当前基于扩散的方法同时生成整段运动序列,限制了其捕捉人类交互中反应性与自适应特性的能力。为解决上述局限,我们提出 Interact2Ar,这是首个用于生成全身体人-人交互的端到端文本条件自回归扩散模型。Interact2Ar 通过专门的并行分支引入细致的手部运动学,从而实现高保真全身体生成。进一步地,我们引入自回归流程并配合一种新颖的记忆技术,借助高效的大上下文窗口适应人类交互固有的多样性。模型的自适应能力使其支持一系列下游应用,包括时序运动组合、对扰动的实时自适应,以及从双人场景扩展到多人场景。为验证生成的运动,我们提出一组稳健的评估器与扩展指标,专门用于评估全身体交互。通过定量与定性实验,我们展示了 Interact2Ar 达到最优性能。
英文原文摘要 1551 字符
Generating realistic human-human interactions is a challenging task that requires not only high-quality individual body and hand motions, but also coherent coordination among all interactants. Due to limitations in available data and increased learning complexity, previous methods tend to ignore hand motions, limiting the realism and expressivity of the interactions. Additionally, current diffusion-based approaches generate entire motion sequences simultaneously, limiting their ability to capture the reactive and adaptive nature of human interactions. To address these limitations, we introduce Interact2Ar, the first end-to-end text-conditioned autoregressive diffusion model for generating full-body, human-human interactions. Interact2Ar incorporates detailed hand kinematics through dedicated parallel branches, enabling high-fidelity full-body generation. Furthermore, we introduce an autoregressive pipeline coupled with a novel memory technique that facilitates adaptation to the inherent variability of human interactions using efficient large context windows. The adaptability of our model enables a series of downstream applications, including temporal motion composition, real-time adaptation to disturbances, and extension beyond dyadic to multi-person scenarios. To validate the generated motions, we introduce a set of robust evaluators and extended metrics designed specifically for assessing full-body interactions. Through quantitative and qualitative experiments, we demonstrate the state-of-the-art performance of Interact2Ar.