ExtrinSplat:解耦几何与语义以实现三维高斯泼溅中的开放词表理解
ExtrinSplat: Decoupling Geometry and Semantics for Open-Vocabulary Understanding in 3D Gaussian Splatting
用多粒度重叠高斯簇加 VLM 生成文本假设的外在索引层,替代逐高斯特征嵌入,可显著降低开放词表三维场景理解的存储与适配时间,适合迁移到任何需要语言查询的三维表示。
原文依据 ExtrinSplat clusters Gaussians into multi-granularity, overlapping 3D object groups.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论将二维开放词表理解提升到三维高斯泼溅(3DGS)场景中是一项关键挑战。主流方法建立在嵌入范式之上,存在三个关键缺陷:(i)几何-语义不一致,即以点而非物体作为语义基础,限制了语义保真度;(ii)语义膨胀,即向几何中注入数 GB 的特征数据;(iii)语义刚性,即每个高斯仅有一个特征,难以捕捉丰富的多义性。为克服这些局限,我们提出 ExtrinSplat,一个建立在外在范式之上、将几何与语义解耦的框架。ExtrinSplat 不嵌入特征,而是将高斯聚类为多粒度、相互重叠的三维物体组。随后由视觉语言模型(VLM)解释这些组,生成轻量化的文本假设,从而构建一个原生支持复杂多义性的外在索引层。通过以轻量索引替代昂贵的特征嵌入,ExtrinSplat 将场景适配时间从数小时缩短至数分钟,并将存储开销降低数个数量级。在开放词表三维物体选择与语义分割的基准任务上,ExtrinSplat 优于已有的基于嵌入的框架,验证了所提出外在范式的有效性与高效性。
英文原文摘要 1337 字符
Lifting 2D open-vocabulary understanding into 3D Gaussian Splatting (3DGS) scenes is a critical challenge. Mainstream methods, built on an embedding paradigm, suffer from three key flaws: (i) geometry-semantic inconsistency, where points, rather than objects, serve as the semantic basis, limiting semantic fidelity; (ii) semantic bloat from injecting gigabytes of feature data into the geometry; and (iii) semantic rigidity, as one feature per Gaussian struggles to capture rich polysemy. To overcome these limitations, we introduce ExtrinSplat, a framework built on the extrinsic paradigm that decouples geometry from semantics. Instead of embedding features, ExtrinSplat clusters Gaussians into multi-granularity, overlapping 3D object groups. A Vision-Language Model (VLM) then interprets these groups to generate lightweight textual hypotheses, creating an extrinsic index layer that natively supports complex polysemy. By replacing costly feature embedding with lightweight indices, ExtrinSplat reduces scene adaptation time from hours to minutes and lowers storage overhead by several orders of magnitude. On benchmark tasks for open-vocabulary 3D object selection and semantic segmentation, ExtrinSplat outperforms established embedding-based frameworks, validating the efficacy and efficiency of the proposed extrinsic paradigm.