无需 3D 扫描的 3D 学习：从视频生成点云进行可扩展预训练
3D sans 3D Scans: Scalable Pre-training from Video-Generated Point Clouds
用前馈重建模型从无标注室内漫游视频生成点云，再以多层级 Sinkhorn-Knopp 聚类加噪声正则损失做自监督预训练，为缺乏 3D 扫描数据时训练点云骨干网络提供了一条可复用的数据与目标函数方案。
原文依据 learns from video-generated point clouds reconstructed from unlabeled videos
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管 3D 自监督学习近期取得了进展，但收集大规模 3D 场景扫描仍然昂贵且耗费人力。在本工作中，我们研究能否从没有任何真实 3D 传感器采集的无标注视频中学习 3D 表征。我们提出 LAM3C（Laplacian-Aware Multi-level 3D Clustering with Sinkhorn-Knopp），这是一个从无标注视频重建得到的视频生成点云上学习的自监督框架。我们首先构建了 RoomTours，这是一个视频生成点云数据集，通过从网络收集房间漫游视频（例如房产导览）并利用现成的前馈重建模型生成 49,219 个场景而构建。我们还提出了一种噪声正则化损失，通过强制局部几何平滑性并确保含噪点云下特征的稳定性，来稳定表征学习。值得注意的是，在不使用任何真实 3D 扫描的情况下，LAM3C 在室内语义分割与实例分割上取得了优于先前自监督方法的性能。这些结果表明，无标注视频是 3D 自监督学习的一个丰富数据来源。我们的源代码见 https://github.com/ryosuke-yamada/lam3c。
英文原文摘要 1228 字符
Despite recent progress in 3D self-supervised learning, collecting large-scale 3D scene scans remains expensive and labor-intensive. In this work, we investigate whether 3D representations can be learned from unlabeled videos recorded without any real 3D sensors. We present Laplacian-Aware Multi-level 3D Clustering with Sinkhorn-Knopp (LAM3C), a self-supervised framework that learns from video-generated point clouds reconstructed from unlabeled videos. We first introduce RoomTours, a video-generated point cloud dataset constructed by collecting room-walkthrough videos from the web (e.g., real-estate tours) and generating 49,219 scenes using an off-the-shelf feed-forward reconstruction model. We also propose a noise-regularized loss that stabilizes representation learning by enforcing local geometric smoothness and ensuring feature stability under noisy point clouds. Remarkably, without using any real 3D scans, LAM3C achieves better performance than previous self-supervised methods on indoor semantic and instance segmentation. These results suggest that unlabeled videos represent an abundant source of data for 3D self-supervised learning. Our source code is available at https://github.com/ryosuke-yamada/lam3c.