生成模型与扩散 · CVPR2026 ·2026.06.03

HandX:扩展双手运动与交互生成

HandX: Scaling Bimanual Motion and Interaction Generation

Zimu Zhang、Yucheng Zhang、Xiyan Xu、Ziyin Wang、Sirui Xu、Kai Zhou 等 11 人

与你有关的部分

对手部精细动作与双手交互生成感兴趣的视觉研究者,可直接复用其解耦标注范式(先抽取接触事件、手指弯曲等运动特征,再用 LLM 生成语义描述)以及手部专用评测指标,用于构建条件运动生成的数据与评测流程。

原文依据 we introduce a decoupled paradigm that extracts representative motion features, e.g., contact events and finger flexion, and then leverages reasoning from large language models
可迁移到 人体运动生成双手交互生成运动捕捉条件生成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

人体运动合成已取得快速进展,但逼真的手部运动与双手交互仍研究不足。全身模型往往遗漏驱动灵巧行为的细粒度线索,即手指关节活动、接触时序与双手协调,且现有资源缺乏能刻画细微手指动态与协作的高保真双手序列。为填补这一空白,我们提出 HandX,一个涵盖数据、标注与评测的统一基础。我们整合并过滤现有数据集以保证质量,并采集了一个新的动作捕捉数据集,聚焦于此前覆盖不足、具有细致手指动态的双手交互。为实现可扩展的标注,我们引入一种解耦范式:先提取代表性运动特征(如接触事件与手指弯曲),再借助大语言模型的推理能力,生成与这些特征对齐的细粒度、语义丰富的描述。基于所得数据与标注,我们对扩散模型与自回归模型在多种条件模式下的表现进行了基准测试。实验表明,在我们新提出的以手部为中心的指标支持下,该方法能够生成高质量的灵巧运动。我们进一步观察到明显的规模扩展趋势:在更大、更高质量数据集上训练的更大模型能生成语义更连贯的双手运动。我们的数据集已公开发布以支持未来研究。

英文原文摘要 1404 字符

Synthesizing human motion has advanced rapidly, yet realistic hand motion and bimanual interaction remain underexplored. Whole-body models often miss the fine-grained cues that drive dexterous behavior, finger articulation, contact timing, and inter-hand coordination, and existing resources lack high-fidelity bimanual sequences that capture nuanced finger dynamics and collaboration. To fill this gap, we present HandX, a unified foundation spanning data, annotation, and evaluation. We consolidate and filter existing datasets for quality, and collect a new motion-capture dataset targeting underrepresented bimanual interactions with detailed finger dynamics. For scalable annotation, we introduce a decoupled paradigm that extracts representative motion features, e.g., contact events and finger flexion, and then leverages reasoning from large language models to produce fine-grained, semantically rich descriptions aligned with these features. Building on the resulting data and annotations, we benchmark diffusion and autoregressive models with versatile conditioning modes. Experiments demonstrate high-quality dexterous motion generation, supported by our newly proposed hand-focused metrics. We further observe clear scaling trends: larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion. Our dataset is released to support future research.

标签

会议 CVPR2026
页码 2274-2284
Bibkey
来源 CVF Open Access