3D 视觉与重建 · CVPR2026 ·2026.06.03

面向室内场景的单目开放词表占用预测

Monocular Open Vocabulary Occupancy Prediction for Indoor Scenes

Changqing Zhou、Yueru Luo、Han Zhang、Zeyu Jiang、Changhao Chen

与你有关的部分

把 3D Language-Embedded Gaussians 当作几何与语言对齐的统一中间表示,用仅二值占用标签的弱监督训练开放词表室内占用预测;其中不透明度感知的 Poisson 体素聚合与渐进温度衰减调度可直接迁移到其他高斯-语言对齐或弱监督高斯到体素转换的流程。

原文依据 we introduce an opacity-aware, Poisson-based approach that stabilizes volumetric aggregation
可迁移到 开放词表三维占用预测单目三维重建开放词表三维语义分割新视角合成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

开放词表三维占用对于具身智能体至关重要,因为这类智能体需要理解语义类别丰富、且会不断演化并超出固定分类体系的复杂室内环境。尽管近期工作已在室外驾驶场景中探索开放词表占用,但这些方法迁移到室内时效果很差:室内几何更稠密、布局更复杂、语义也精细得多。为应对这些挑战,我们采用仅依赖几何的监督范式,即只使用二值占用标签(占用与空闲)。我们的框架建立在三维语言嵌入高斯之上,这种表示作为统一的中间表示,将精细的三维几何与语言对齐的语义嵌入耦合在一起。在几何方面,我们发现现有的高斯到占用算子在这种弱监督下无法收敛,于是引入一种不透明度感知的、基于泊松的方法来稳定体素聚合。在语义方面,渲染特征与开放词表分割特征之间的直接对齐会受到特征混合的影响;因此我们提出渐进温度衰减调度,在泼溅过程中逐步锐化不透明度,从而增强高斯与语言之间的对齐。在 Occ-ScanNet 上,我们的框架在开放词表设定下取得 59.50 的 IoU 和 21.05 的平均交并比,在 IoU 上超过所有现有占用方法,并在平均交并比上大幅领先此前的开放词表方法。代码将发布于 https://github.com/JuIvyy/LegoOcc。

英文原文摘要 1526 字符

Open-vocabulary 3D occupancy is vital for embodied agents, which need to understand complex indoor environments where semantic categories are abundant and evolve beyond fixed taxonomies. While recent work has explored open-vocabulary occupancy in outdoor driving scenarios, such methods transfer poorly indoors, where geometry is denser, layouts are more intricate, and semantics are far more fine-grained. To address these challenges, we adopt a geometry-only supervision paradigm that uses only binary occupancy labels (occupied vs. free). Our framework builds upon 3D Language-Embedded Gaussians, which serve as a unified intermediate representation coupling fine-grained 3D geometry with a language-aligned semantic embedding. On the geometry side, we find that existing Gaussian-to-Occupancy operators fail to converge under such weak supervision, and we introduce an opacity-aware, Poisson-based approach that stabilizes volumetric aggregation. On the semantic side, direct alignment between rendered features and open-vocabulary segmentation features suffers from feature mixing; we therefore propose a Progressive Temperature Decay schedule that gradually sharpens opacities during splatting, strengthening Gaussian-language alignment. On Occ-ScanNet, our framework achieves 59.50 IoU and 21.05 mIoU in the open-vocabulary setting, surpassing all existing occupancy methods in IoU and outperforming prior open-vocabulary approaches by a large margin in mIoU. Code will be released at https://github.com/JuIvyy/LegoOcc.

标签

会议 CVPR2026
页码 21627-21637
Bibkey
来源 CVF Open Access