3D 视觉与重建 · CVPR2026 ·2026.06.03

GeoSAM2:释放 SAM2 在三维部件分割中的能力

GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation

Ken Deng、Yunhan Yang、Jingxiang Sun、Xihui Liu、Yebin Liu、Ding Liang 等 7 人

与你有关的部分

把 3D 部件分割转成多视图 2D 掩码预测,用 SAM2 加低秩适配和残差几何融合,接受点击/框提示,再把掩码反投影聚合到物体上;做 3D 分割或想复用 SAM2 先验的人可直接借鉴这套提示可控、免逐形状优化的流程。

原文依据 casts the task as multi-view 2D mask prediction
可迁移到 3D 部件分割多视图分割提示驱动分割三维重建

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

我们提出 GeoSAM2,一个提示可控的三维部件分割框架,将任务转化为多视图二维掩码预测。给定一个无纹理物体,我们从预定义视角渲染法线图与点图,并接受简单的二维提示——点击或框——来引导部件选择。这些提示由共享的 SAM2 骨干网络处理,该骨干网络经低秩适配与残差几何融合增强,从而在保留预训练先验的同时实现视角特定的推理。预测的掩码被反投影到物体上,并在各视图间聚合。我们的方法无需文本提示、逐形状优化或完整三维标注,即可实现细粒度、部件特定的控制。与全局聚类或基于尺度的方法相比,提示是显式的、空间上有依据的且可解释的。我们在 PartObjaverse-Tiny 与 PartNetE 上取得最优的类别无关性能,优于缓慢的基于优化的流程以及快速但粗糙的前馈方法。我们的结果凸显了一种新范式:使三维分割的范式与 SAM2 对齐,利用交互式二维输入释放物体级部件理解中的可控性与精度。

英文原文摘要 1177 字符

We introduce GeoSAM2, a prompt-controllable framework for 3D part segmentation that casts the task as multi-view 2D mask prediction. Given a textureless object, we render normal and point maps from predefined viewpoints and accept simple 2D prompts--clicks or boxes--to guide part selection. These prompts are processed by a shared SAM2 backbone augmented with LoRA and residual geometry fusion, enabling view-specific reasoning while preserving pretrained priors. The predicted masks are back-projected to the object, aggregated across views.Our method enables fine-grained, part-specific control without requiring text prompts, per-shape optimization, or full 3D labels. In contrast to global clustering or scale-based methods, prompts are explicit, spatially grounded, and interpretable. We achieve state-of-the-art class-agnostic performance on PartObjaverse-Tiny and PartNetE, outperforming both slow optimization-based pipelines and fast but coarse feedforward approaches. Our results highlight a new paradigm: aligning the paradigm of 3D segmentation with SAM2, leveraging interactive 2D inputs to unlock controllability and precision in object-level part understanding.

标签

会议 CVPR2026
页码 6367-6376
Bibkey
来源 CVF Open Access