面向基于视觉人机交互的深度与运动融合实时多模态指尖接触检测
Real-Time Multimodal Fingertip Contact Detection via Depth and Motion Fusion for Vision-Based Human-Computer Interaction
用专用 RGB-D 数据微调现成深度估计模型，把毫米级深度误差压到 3.8 mm，从而区分悬停与真实接触；这套「小规模专用数据 + 深度模型微调」的流程可直接迁移到其他需要亚厘米精度的手部/人体交互感知任务。
原文依据 By systematically fine-tuning six state-of-the-art depth estimation architectures on this dataset, we reduce the mean absolute error (MAE) by 68%
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论精确的指尖接触检测是实现自然、沉浸式虚拟现实（VR）交互的一项基础性挑战。然而，现有的基于视觉的方法精度不足，典型的深度误差（12–25 mm）过大，无法可靠区分悬停与真实接触（<3 mm）。商用动作捕捉系统虽能提供亚毫米级精度，但其高昂成本限制了广泛应用。本文针对这一关键空白，开发了一套高精度且低成本的指尖接触检测系统。我们构建了一个全新的专用数据集，包含 53,300 对 RGB-深度图像，记录了毫米级的手-桌面打字交互。通过在该数据集上系统性地微调六种最优的深度估计架构，我们将平均绝对误差（MAE）降低了 68%，从 12.3 mm 降至最优的 3.8 mm。我们完整的 VR 键盘系统 TapBoard-X 实现了 95.96% 的接触检测准确率，支持 45.6 WPM 的打字速度，字符错误率低至 3.1%，可与物理键盘相媲美。该性能在成本相比商用系统降低 90% 以上的条件下实现，使高精度手部跟踪得以惠及更广泛的研究群体，并为下一代触觉 VR 体验铺平了道路。
英文原文摘要 1320 字符
Precise fingertip contact detection is a fundamental challenge for natural and immersive virtual reality (VR) interaction. However, existing vision-based methods suffer from insufficient accuracy, with typical depth errors (12-25 mm) being too large to reliably distinguish between hovering and true contact (<3 mm). While commercial motion capture systems provide sub-millimeter accuracy, their prohibitive cost limits widespread adoption. This paper addresses this critical gap by developing a highly accurate and cost-effective system for fingertip contact detection. We introduce a novel, specialised dataset of 53,300 RGB-depth pairs capturing millimeter-scale, hand-table typing interactions. By systematically fine-tuning six state-of-the-art depth estimation architectures on this dataset, we reduce the mean absolute error (MAE) by 68%, from 12.3 mm to a state-of-the-art 3.8 mm. Our complete VR keyboard system, TapBoard-X, achieves 95.96% contact detection accuracy and enables typing speeds of 45.6 WPM with a low 3.1% character error rate, rivalling physical keyboards. This performance is achieved at over a 90% cost reduction compared to commercial systems, democratising high-precision hand tracking for the broader research community and paving the way for the next generation of tactile VR experiences.