CoSMo3D：通过 LLM 引导的规范空间建模实现开放世界提示式 3D 语义分割
CoSMo3D: Open-World Promptable 3D Semantic Segmentation through LLM-Guided Canonical Spatial Modeling
把 3D 语义分割从输入传感器位姿空间搬到数据驱动的规范参考系中，用 LLM 引导构建跨类别规范数据集、再用规范图锚定与规范框校准的双分支把位姿与对称性坍缩成稳定嵌入——这套「先规范再分割」的思路可直接迁移到任意位姿敏感的点云/网格部件分割与开放词表 3D 任务。
原文依据 we propose CoSMo3D, which attains canonical space perception by inducing a latent canonical reference frame learned directly from data
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论开放世界提示式 3D 语义分割仍然脆弱，因为语义是在输入传感器坐标系中推断的。然而与之相反，人类通过物体在规范空间中的功能角色来理解部件——机翼向两侧伸展、把手向侧面突出、支腿从下方支撑。心理物理学证据表明，我们会在头脑中将物体旋转到规范坐标系以揭示这些角色。为填补这一空白，我们提出 CoSMo3D，它通过引入一个直接从数据中学习得到的潜变量规范参考系来实现规范空间感知。在构造层面，我们通过 LLM 引导的类内与跨类对齐构建了一个统一的规范数据集，在 200 个类别上展现出规范空间规律性。在归纳层面，我们通过带有规范图锚定与规范框校准的双分支架构在模型内部实现规范性，将位姿变化与对称性坍缩为一个稳定的规范嵌入。这种从输入位姿空间到规范表示的转变带来了远为稳定且可迁移的部件语义。实验结果表明，CoSMo3D 在开放世界提示式 3D 分割上建立了新的最先进水平。
英文原文摘要 1181 字符
Open-world promptable 3D semantic segmentation remains brittle as semantics are inferred in the input sensor coordinates. Yet, humans, in contrast, interpret parts via functional roles in a canonical space -- wings extend laterally, handles protrude to the side, and legs support from below. Psychophysical evidence shows that we mentally rotate objects into canonical frames to reveal these roles. To fill this gap, we propose CoSMo3D, which attains canonical space perception by inducing a latent canonical reference frame learned directly from data. By construction, we create a unified canonical dataset through LLM-guided intra- and cross-category alignment, exposing canonical spatial regularities across 200 categories. By induction, we realize canonicality inside the model through a dual-branch architecture with canonical map anchoring and canonical box calibration, collapsing pose variation and symmetry into a stable canonical embedding. This shift from input pose space to canonical representation yields far more stable and transferable part semantics. Experimental results show that CoSMo3D establishes new state of the art in open-world promptable 3D segmentation.