PAD-Hand:面向手部运动恢复的物理感知扩散模型
PAD-Hand: Physics-Aware Diffusion for Hand Motion Recovery
把欧拉-拉格朗日动力学残差当作虚拟观测,接入条件扩散模型来精修手部姿态序列,并用最后一层拉普拉斯近似给出逐关节、逐帧的物理一致性方差;做手部或人体运动捕捉时,可直接复用这套「物理残差+不确定性」的后处理精修思路。
原文依据 we treat the resulting dynamic residuals as virtual observables to more effectively integrate physics
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从图像重建手部的研究已取得显著进展,能够给出准确的单帧估计,但这些估计往往缺乏物理一致性,也无法说明运动在多大程度上满足物理规律。本文提出一种新的物理感知条件扩散框架,将含噪的姿态序列精修为物理上合理的手部运动,同时估计运动估计中的物理方差。我们在 MeshCNN-Transformer 主干之上,为关节式手部构建欧拉-拉格朗日动力学。与以往强制残差为零的工作不同,我们将得到的动力学残差视为虚拟观测,从而更有效地融入物理约束。通过最后一层拉普拉斯近似,本方法给出逐关节、逐时刻的方差,用以度量物理一致性,并提供可解释的方差图,指示物理一致性在何处减弱。在两个知名手部数据集上的实验表明,本方法相对强力的基于图像的初始化以及有竞争力的基于视频的方法均取得稳定提升。定性结果证实,我们的方差估计与基于图像的估计中运动的物理合理性相一致。
英文原文摘要 1187 字符
Significant advancements made in reconstructing hands from images have delivered accurate single-frame estimates, yet they often lack physics consistency and provide no notion of how confidently the motion satisfies physics. In this paper, we propose a novel physics-aware conditional diffusion framework that refines noisy pose sequences into physically plausible hand motion while estimating the physics variance in motion estimates. Building on a MeshCNN-Transformer backbone, we formulate Euler-Lagrange dynamics for articulated hands. Unlike prior works that enforce zero residuals, we treat the resulting dynamic residuals as virtual observables to more effectively integrate physics. Through a last-layer Laplace approximation, our method produces per-joint, per-time variances that measure physics consistency and offers interpretable variance maps indicating where physical consistency weakens. Experiments on two well-known hand datasets show consistent gains over strong image-based initializations and competitive video-based methods. Qualitative results confirm that our variance estimations are aligned with the physical plausibility of the motion in image-based estimates.