Glove2Hand：从多模态传感手套合成自然手物交互
Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Gloves
用多模态传感手套（触觉+IMU）驱动3D高斯手模型，再经扩散式手部修复器渲染出无遮挡的裸手视频，为严重遮挡下的手部跟踪与接触力估计提供可复用的数据合成与监督信号来源。
原文依据 a framework that translates multi-modal sensing glove data in HOI videos into photorealistic bare-hand representations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论理解手物交互（HOI）是计算机视觉、机器人学以及 AR/VR 的基础。然而，传统的手部视频往往缺乏接触力与运动动力学等关键物理信息，并且容易出现频繁遮挡。为应对这些挑战，我们提出 Glove2Hand，一个将 HOI 视频中的多模态传感手套数据转换为照片级真实裸手表示的框架，同时忠实保留其背后的物理交互动力学。我们引入一种新颖的 3D 高斯手模型，确保时序与多视角渲染的一致性。渲染出的手通过基于扩散的手部修复器无缝融入场景，该修复器能有效处理复杂的手物交互与非刚性形变。借助 Glove2Hand，我们推出 HandSense，这是首个包含多视角裸手视频并同步触觉与 IMU 信号的多模态 HOI 数据集。我们证明 HandSense 显著提升了裸手下游应用，包括基于视频的接触估计与严重遮挡下的手部跟踪。
英文原文摘要 1130 字符
Understanding hand-object interaction (HOI) is fundamental to computer vision, robotics, and AR/VR. However, conventional hand videos often lack essential physical information, such as contact forces and motion dynamics, and are prone to frequent occlusions. To address these challenges, we present Glove2Hand, a framework that translates multi-modal sensing glove data in HOI videos into photorealistic bare-hand representations, while faithfully preserving the underlying physical interaction dynamics. We introduce a novel 3D Gaussian hand model that ensures both temporal and multi-view rendering consistency. The rendered hand is seamlessly integrated into the scene using a diffusion-based hand restorer, which effectively handles complex hand-object interactions and non-rigid deformations. Leveraging Glove2Hand, we introduce HandSense, the first multi-modal HOI dataset featuring multi-view bare-hand videos with synchronized tactile and IMU signals. We demonstrate that HandSense significantly enhances downstream bare-hand applications, including video-based contact estimation and hand tracking under severe occlusion.