SAMosaic3D：面向实时三维分割任意物体的模块化场景组装
SAMosaic3D: Modular Scene Assembly for Real-Time 3D Segment Anything
将 SAM 的 2D 掩码视为可学习的「马赛克瓦片」，通过可微的片段到实例自适应组装和实例到场景在线合并，解决 2D 提升到 3D 时的空间碎片化与身份漂移问题，为在线 3D 实例分割提供了可复用的模块化组装框架。
原文依据 we reframe online segmentation as a learnable composition problem
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在线三维实例分割是具身智能体在动态环境中导航的关键能力。然而，将强大的二维基础模型（如 SAM）适配到三维在线分割仍面临根本性挑战。简单地将 SAM 的二维掩码提升到三维会导致严重的空间碎片化，单个物体会被分裂成多个不连通的部分，尤其在遮挡情况下。后续尝试通过传统的基于三维 IoU 的跟踪来在时间上链接这些碎片，被证明非常脆弱：它们难以处理遮挡或拓扑变化，最终导致灾难性的身份漂移。与这类后处理方法不同，我们将在线分割重新定义为可学习的组合问题。我们提出 SAMosaic3D，一个可微框架，将 SAM 衍生的掩码视为“马赛克瓦片”，并学习将它们组装成时间上一致的三维实例。SAMosaic3D 包含两个关键组件：片段到实例的自适应组装，通过软门控注意力聚合片段；以及实例到场景的在线合并，采用级联的语义-几何匹配，通过学习到的短期关联保持物体身份，并通过几何验证进行长期重识别。在 ScanNet、ScanNet200、SceneNN 和 3RScan 数据集上的评估展示了最优性能和零样本跨数据集泛化能力。大量消融研究验证了所设计模块的有效性。
英文原文摘要 1494 字符
Online 3D instance segmentation is a critical capability for embodied agents navigating in dynamic environments. However, a fundamental challenge remains in adapting powerful 2D foundation models, like SAM, to 3D online segmentation. Naively lifting SAM's 2D masks to 3D results in severe spatial fragmentation, where a single object is shattered into multiple disconnected parts, especially under occlusion. Subsequent attempts to link these fragments over time via conventional 3D IoU-based tracking prove highly fragile: they struggle to handle occlusions or topological changes, ultimately causing catastrophic identity drift. Departing from such post-processing approaches, we reframe online segmentation as a learnable composition problem. We introduce SAMosaic3D, a differentiable framework that treats SAM-derived masks as "mosaic tiles" and learns to assemble them into temporally consistent 3D instances. SAMosaic3D comprises two key components: Fragment-to-Instance Adaptive Assembly that aggregates fragments through soft-gated attention, and Instance-to-Scene Online Merging that employs cascaded semantic-geometric matching to preserve object identities through learned short-term association and geometric verification for long-term re-identification. Evaluations on ScanNet, ScanNet200, SceneNN and 3RScan datasets demonstrate state-of-the-art performance and zero-shot cross-dataset generalization. Extensive ablation studies validate the effectiveness of the designed modules.