以条件手部调制器增强三维全身姿态估计中的手部
Enhancing Hands in 3D Whole-Body Pose Estimation with Conditional Hands Modulator
用预训练手部估计器的特征通过轻量调制器(CHAM)注入全身姿态网络的特征流,无需重训全身模型即可修正手腕朝向;手指关节与手形则经可微刚性对齐贴到全身网格上,这套「冻结主干+外挂专家调制」的模块化思路可直接迁移到其他部位精度不足的全身/多人重建任务。
原文依据 a lightweight module that modulates the whole-body feature stream using hand-specific features extracted from a pre-trained hand pose estimator
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在身体上下文中准确恢复手部姿态,仍是三维全身姿态估计中的主要挑战。这一困难源于根本性的监督鸿沟:全身姿态估计器在全身数据集上训练,手部多样性有限;而仅在以手为中心的数据集上训练的手部估计器虽擅长精细手指关节建模,却缺乏全局身体感知。为此,我们提出 Hand4Whole++,一个模块化框架,融合预训练的全身姿态估计器与手部姿态估计器各自的优势。我们引入 CHAM(条件手部调制器),这是一个轻量模块,利用从预训练手部姿态估计器中提取的手部特定特征来调制全身特征流。该调制使全身模型能够预测既准确又与上半身运动学结构一致的手腕朝向,且无需重新训练全身模型。同时,我们直接采用手部姿态估计器预测的手指关节与手形,通过可微刚性对齐将其对齐到全身网格。这一设计使 Hand4Whole++ 能够将全局一致的身体推理与细粒度手部细节结合起来。大量实验表明,Hand4Whole++ 显著提升了手部精度,并改善了整体全身姿态质量。
英文原文摘要 1358 字符
Accurately recovering hand poses within the body context remains a major challenge in 3D whole-body pose estimation. This difficulty arises from a fundamental supervision gap: whole-body pose estimators are trained on full-body datasets with limited hand diversity, while hand-only estimators, trained on hand-centric datasets, excel at detailed finger articulation but lack global body awareness. To address this, we propose Hand4Whole++, a modular framework that leverages the strengths of both pre-trained whole-body and hand pose estimators. We introduce CHAM (Conditional Hands Modulator), a lightweight module that modulates the whole-body feature stream using hand-specific features extracted from a pre-trained hand pose estimator. This modulation enables the whole-body model to predict wrist orientations that are both accurate and coherent with the upper-body kinematic structure, without retraining the full-body model. In parallel, we directly incorporate finger articulations and hand shapes predicted by the hand pose estimator, aligning them to the full-body mesh via differentiable rigid alignment. This design allows Hand4Whole++ to combine globally consistent body reasoning with fine-grained hand detail. Extensive experiments demonstrate that Hand4Whole++ substantially improves hand accuracy and enhances overall full-body pose quality.