OccAny:广义无约束城市三维占用
OccAny: Generalized Unconstrained Urban 3D Occupancy
OccAny 将视觉几何基础模型适配到城市场景的度量占用预测,并引入分割强制和新视角渲染实现测试时视图增强,为无标定场景下的三维占用补全提供了可复用的框架。
原文依据 the first unconstrained urban 3D occupancy model capable of operating on out-of-domain uncalibrated scenes
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有的三维占用预测方法依赖域内标注和精确的传感器装配先验,在可扩展性和域外泛化方面均受限。尽管近期的视觉几何基础模型展现出强大的泛化能力,但它们主要面向通用目的设计,缺乏城市占用预测所需的一个或多个关键要素,即度量预测、杂乱场景中的几何补全以及对城市场景的适应能力。我们填补这一空白并提出 OccAny,这是首个无约束城市三维占用模型,能够在域外未标定场景上运行,预测并补全度量占用及分割特征。OccAny 具有通用性,可从序列、单目或环视图像预测占用。我们的贡献有三方面:(i) 我们提出了首个广义三维占用框架,并包含 (ii) 分割强制(Segmentation Forcing),在提升占用质量的同时实现掩码级预测,以及 (iii) 新视角渲染流程,推断新视角几何以实现测试时视图增强以完成几何补全。大量实验表明,OccAny 在三维占用预测任务上优于所有视觉几何基线,同时在两个已建立的城市占用预测数据集上,在三种输入设置下与域内自监督方法保持竞争力。我们的代码可在 https://github.com/valeoai/OccAny 获取。
英文原文摘要 1467 字符
Relying on in-domain annotations and precise sensor-rig priors, existing 3D occupancy prediction methods are limited in both scalability and out-of-domain generalization. While recent visual geometry foundation models exhibit strong generalization capabilities, they were mainly designed for general purposes and lack one or more key ingredients required for urban occupancy prediction, namely metric prediction, geometry completion in cluttered scenes and adaptation to urban scenarios. We address this gap and present OccAny, the first unconstrained urban 3D occupancy model capable of operating on out-of-domain uncalibrated scenes to predict and complete metric occupancy coupled with segmentation features. is versatile and can predict occupancy from sequential, monocular, or surround-view images. Our contributions are three-fold: (i) we propose the first generalized 3D occupancy framework with (ii)Segmentation Forcing that improves occupancy quality while enabling mask-level prediction, and (iii) a Novel View Rendering pipeline that infers novel-view geometry to enable test-time view augmentation for geometry completion. Extensive experiments demonstrate that OccAny outperforms all visual geometry baselines on 3D occupancy prediction task, while remaining competitive with in-domain self-supervised methods across three input settings on two established urban occupancy prediction datasets. Our code is available at https://github.com/valeoai/OccAny .