利用互联网级无标注数据提升 3D 场景理解
Lifting Unlabeled Internet-level Data for 3D Scene Understanding
提出一种数据引擎，利用互联网无标注视频自动生成 3D 场景理解训练数据，可复用于低层感知（3D 检测/分割）与高层推理（3D VQA/VLN）任务，为缓解 3D 标注稀缺提供可扩展方案。
原文依据 carefully designed data engines can leverage web-curated, unlabeled videos to automatically generate training data
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论标注的 3D 场景数据稀缺且获取成本高昂，而互联网上存在大量无标注视频。本文证明，精心设计的数据引擎能够利用网络收集的无标注视频自动生成训练数据，从而与人工标注数据集一起促进 3D 场景理解的端到端模型。我们识别并分析了自动数据生成中的瓶颈，揭示了决定从无标注数据中学习效率与效果的关键因素。为了在不同感知粒度上验证我们的方法，我们在三个任务上进行了评估，涵盖低层感知（即 3D 目标检测和实例分割）到高层推理（即 3D 空间视觉问答（VQA）和视觉语言导航（VLN））。在我们生成的数据上训练的模型展现出强大的零样本性能，并在微调后进一步提升。这表明利用现成的网络数据是通向更强大场景理解系统的一条可行路径。
英文原文摘要 1097 字符
Annotated 3D scene data is scarce and expensive to acquire, while abundant unlabeled videos are readily available on the internet. In this paper, we demonstrate that carefully designed data engines can leverage web-curated, unlabeled videos to automatically generate training data, to facilitate end-to-end models in 3D scene understanding alongside human-annotated datasets. We identify and analyze bottlenecks in automated data generation, revealing critical factors that determine the efficiency and effectiveness of learning from unlabeled data. To validate our approach across different perception granularities, we evaluate on three tasks spanning low-level perception, i.e., 3D object detection and instance segmentation, to high-level reasoning, i.e., 3D spatial Visual Question Answering (VQA) and Vision-Lanugage Navigation (VLN). Models trained on our generated data demonstrate strong zero-shot performance and show further improvement after finetuning. This demonstrates the viability of leveraging readily available web data as a path toward more capable scene understanding systems.