SHOW3D：在野外捕获3D手与物体场景
SHOW3D: Capturing Scenes of 3D Hands and Objects in the Wild
提供了一套可移动的多相机采集系统与ego-exo跟踪流程，用于在真实野外环境中生成手与物体的精确3D标注，可直接用于训练和评测第一人称手物交互理解模型。
原文依据 we introduce a novel marker-less multi-camera system that allows for nearly unconstrained mobility in genuinely in-the-wild conditions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在操作过程中对人类手部和物体进行准确的3D理解仍然是第一人称计算机视觉的一项重大挑战。现有的手物交互数据集主要在受控的摄影棚环境中采集，这限制了环境多样性以及在此类数据上训练的模型对真实场景的泛化能力。为解决这一挑战，我们提出了一种新颖的无标记多相机系统，能够在真正野外的条件下实现几乎无约束的移动性，同时仍能生成手部和物体的精确3D标注。该采集系统由一个轻量级、背部安装的多相机装置组成，该装置与用户佩戴的VR头显同步并校准。对于手部和物体的3D真值标注，我们开发了一套第一人称-第三人称跟踪流程，并严格评估了其质量。最后，我们提出了SHOW3D，这是首个具有3D标注的大规模数据集，展示了手部在不同真实环境中与物体交互，包括户外场景。我们的方法显著减少了环境真实性与3D标注准确性之间的根本权衡，我们通过在多个下游任务上的实验验证了这一点。该数据集将在接收后公开发布。show3d-dataset.github.io
英文原文摘要 1337 字符
Accurate 3D understanding of human hands and objects during manipulation remains a significant challenge for egocentric computer vision. Existing hand-object interaction datasets are predominantly captured in controlled studio settings, which limits both environmental diversity and the ability of models trained on such data to generalize to real-world scenarios. To address this challenge, we introduce a novel marker-less multi-camera system that allows for nearly unconstrained mobility in genuinely in-the-wild conditions, while still having the ability to generate precise 3D annotations of hands and objects. The capture system consists of a lightweight, back-mounted, multi-camera rig that is synchronized and calibrated with a user-worn VR headset. For 3D ground-truth annotation of hands and objects, we develop an ego-exo tracking pipeline and rigorously evaluate its quality. Finally, we present SHOW3D, the first large-scale dataset with 3D annotations that show hands interacting with objects in diverse real-world environments, including outdoor settings. Our approach significantly reduces the fundamental trade-off between environmental realism and accuracy of 3D annotations, which we validate with experiments on several downstream tasks. The dataset will be publicly released upon acceptance. show3d-dataset.github.io