MGDHand：面向稀疏 IMU 序列三维手部姿态估计的多粒度先验到惯性蒸馏框架
MGDHand: Multi-Granularity Prior-to-Inertial Distillation Framework for Sequential 3D Hand Pose Estimation from Sparse IMUs
把视觉侧 MANO 融合教师模型的静态形状、动态关节运动学与速度/加速度先验，按粒度解耦蒸馏到稀疏 IMU 学生模型，为视觉-惯性跨模态蒸馏提供了可复用的「按信息密度分层对齐」思路，可直接迁移到人体姿态、动捕等稀疏传感器设定。
原文依据 a Multi-Granularity Decoupled Distillation (MGDistill) scheme is proposed to bridge the semantic gap
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于稀疏惯性测量单元（IMU）的三维手部姿态估计在人机交互中展现出巨大潜力。然而，由于稀疏的局部运动信息与结构化的全局姿态信息之间存在显著的语义鸿沟，从稀疏 IMU 信号估计手部姿态具有歧义性且颇具挑战。知识蒸馏能够将更强教师模型中的丰富知识迁移至学生模型，从而提升学生模型的性能。现有方法将形态先验蒸馏到基于 IMU 的学生模型中，有效提升了其在复杂场景下的精度。然而，这些方法忽视了视觉-惯性固有的语义失配与信息密度差异，导致学生模型难以学习耦合的先验。本文提出一种面向稀疏 IMU 序列三维手部姿态估计的多粒度先验到惯性蒸馏框架（MGDHand）。我们首先预训练一个 MANO-IMU 融合模型作为教师，用以编码静态几何形态先验、动态运动学先验与时序运动先验。随后，提出多粒度解耦蒸馏（MGDistill）方案以弥合语义鸿沟。MGDistill 包含一个静态形状蒸馏模块，用于迁移时不变的手部形状先验；以及一个动态姿态蒸馏模块，用于迁移复杂的关节运动学与稠密姿态先验。此外，一个时序运动蒸馏模块用于迁移快速变化的运动先验（速度与加速度）。在公开数据集上的大量实验表明，我们的方法在稀疏 IMU 条件下显著优于最优方法。
英文原文摘要 1642 字符
3D hand pose estimation (HPE) from sparse inertial measurement units (IMUs) has shown great potential in human-computer interaction. However, due to the significant semantic gap between sparse local motion information and structured global pose information, estimating hand poses from sparse IMU signals is ambiguous and challenging. Knowledge distillation can transfer rich knowledge from the stronger teacher to the student model, so that the student enhances performance. Existing approaches distill morphological priors into the IMU-based student model, effectively improving its accuracy in complex scenarios. Nevertheless, overlooking the visual-inertial inherent semantic mismatch and information density difference leads to difficulties for students to learn coupled priors. In this paper, we propose a Multi-Granularity Prior-to-Inertial Distillation Framework for Sequential 3D HPE from Sparse IMUs (MGDHand). We first pre-train a MANO-IMU fusion model as a teacher to encode static geometric morphology prior, dynamic kinematic prior and temporal motion prior. Then, a Multi-Granularity Decoupled Distillation (MGDistill) scheme is proposed to bridge the semantic gap. MGDistill includes a Static Shape Distillation module to transfer time-invariant hand shape priors, and a Dynamic Pose Distillation module to transfer complex joint kinematics and dense pose priors. Additionally, a Temporal Motion Distillation module transfers the fast-changing motion priors (velocity and acceleration). Extensive experiments on public dataset demonstrate that our method significantly outperforms state-of-the-art approaches under sparse IMUs.