MV3DIS：基于 3D 引导的多视图掩码匹配用于零样本 3D 实例分割
MV3DIS: Multi-View Mask Matching via 3D Guides for Zero-Shot 3D Instance Segmentation
用粗粒度 3D 分割结果作为跨视图 2D 掩码匹配的公共参照，并以 3D 覆盖分布与深度一致性加权来筛选多视图掩码，可复用于任何需要把 SAM 类 2D 掩码提升到 3D 的零样本流程。
原文依据 a 3D-guided mask matching strategy that uses coarse 3D segments as a common reference to match 2D masks across views
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论传统的 3D 实例分割方法依赖耗费大量人力的 3D 标注进行监督训练，这限制了其可扩展性以及对新物体的泛化能力。近期方法利用 Segment Anything Model（SAM）产生的多视图 2D 掩码来引导 3D 几何基元的合并，从而实现零样本 3D 实例分割。然而，这些方法通常独立处理每一帧，并仅依赖 SAM 预测分数等 2D 指标来生成分割图。这种设计忽略了多视图之间的关联与固有的 3D 先验，导致不同视图间的 2D 掩码不一致，最终使 3D 分割结果碎片化。本文提出 MV3DIS，一个显式引入 3D 先验的由粗到细零样本 3D 实例分割框架。具体而言，我们引入一种 3D 引导的掩码匹配策略，以粗粒度 3D 分割段作为公共参照来匹配跨视图的 2D 掩码，并通过 3D 覆盖分布来强化多视图掩码的一致性。在这些视图一致的 2D 掩码引导下，粗粒度 3D 分割段被进一步细化为精确的 3D 实例。此外，我们引入一种深度一致性加权方案，用于量化投影的可靠性，以抑制物体间遮挡带来的歧义，从而提升 3D 到 2D 对应关系的鲁棒性。在 ScanNetV2、ScanNet200、ScanNet++、Replica 和 Matterport3D 数据集上的大量实验证明了 MV3DIS 的有效性，其性能优于以往方法。
英文原文摘要 1569 字符
Conventional 3D instance segmentation methods rely on labor-intensive 3D annotations for supervised training, which limits their scalability and generalization to novel objects. Recent approaches leverage multi-view 2D masks from the Segment Anything Model (SAM) to guide the merging of 3D geometric primitives, thereby enabling zero-shot 3D instance segmentation. However, these methods typically process each frame independently and rely solely on 2D metrics, such as SAM prediction scores, to produce segmentation maps. This design overlooks multi-view correlations and inherent 3D priors, leading to inconsistent 2D masks across views and ultimately fragmented 3D segmentation. In this paper, we propose MV3DIS, a coarse-to-fine framework for zero-shot 3D instance segmentation that explicitly incorporates 3D priors. Specifically, we introduce a 3D-guided mask matching strategy that uses coarse 3D segments as a common reference to match 2D masks across views and consolidates multi-view mask consistency via 3D coverage distributions. Guided by these view-consistent 2D masks, the coarse 3D segments are further refined into precise 3D instances. Additionally, we introduce a depth consistency weighting scheme that quantifies projection reliability to suppress ambiguities from inter-object occlusions, thereby improving the robustness of 3D-to-2D correspondence. Extensive experiments on the ScanNetV2, ScanNet200, ScanNet++, Replica, and Matterport3D datasets demonstrate the effectiveness of MV3DIS, which achieves superior performance over previous methods.