OVI-MAP:开放词表实例语义建图
OVI-MAP: Open-Vocabulary Instance-Semantic Mapping
将实例重建与语义推理解耦:用 RGB-D 增量构建类别无关的三维实例图,仅在自动挑选的少量关键帧上调用视觉语言模型提取语义特征,从而在实时在线建图中同时获得稳定的实例跟踪与零样本开放词表标注,可作为开放词表三维场景理解系统的可复用架构。
原文依据 decouples instance reconstruction from semantic inference
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论增量式开放词表三维实例语义建图对于在复杂日常环境中运行的自主智能体至关重要。然而,由于需要鲁棒的实例分割、实时处理以及灵活的开放集推理,该任务仍具挑战性。现有方法往往依赖闭集假设或逐像素的稠密语言融合,限制了可扩展性与时序一致性。本文提出 OVI-MAP,将实例重建与语义推理解耦。我们提出构建一个类别无关的三维实例图,由 RGB-D 输入增量式构建,而语义特征仅借助视觉语言模型从一小组自动选取的视图中提取。该设计使得在整个在线探索过程中能够实现稳定的实例跟踪与零样本语义标注。本系统可实时运行,并在标准基准上优于最优的开放词表建图基线方法。
英文原文摘要 960 字符
Incremental open-vocabulary 3D instance-semantic mapping is essential for autonomous agents operating in complex everyday environments. However, it remains challenging due to the need for robust instance segmentation, real-time processing, and flexible open-set reasoning. Existing methods often rely on the closed-set assumption or dense per-pixel language fusion, which limits scalability and temporal consistency. We introduce OVI-MAP that decouples instance reconstruction from semantic inference. We propose to build a class-agnostic 3D instance map that is incrementally constructed from RGB-D input, while semantic features are extracted only from a small set of automatically selected views using vision-language models. This design enables stable instance tracking and zero-shot semantic labeling throughout online exploration. Our system operates in real time and outperforms state-of-the-art open-vocabulary mapping baselines on standard benchmarks.