OpenVoxel：面向开放词表三维场景理解的免训练体素分组与描述生成
OpenVoxel: Training-Free Grouping and Captioning Voxels for Open-Vocabulary 3D Scene Understanding
对做开放词表 3D 场景理解的视觉研究者，这篇论文提供了一条免训练路径：直接在稀疏体素光栅化（SVR）模型的分组结果上，用 MLLM 做文本到文本检索，替代 CLIP/BERT 文本嵌入，可复用于 OVS/RES 等 3D 任务。
原文依据 our method is training-free and does not introduce embeddings from a CLIP/BERT text encoder
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 OpenVoxel，一种用于开放词表三维场景理解任务的免训练稀疏体素分组与描述生成算法。给定由三维场景的多视图图像得到的稀疏体素光栅化（SVR）模型，OpenVoxel 能够生成描述场景中不同物体的有意义的组。此外，借助强大的视觉语言模型（VLM）与多模态大语言模型（MLLM），OpenVoxel 通过对每个组生成描述，成功构建了信息丰富的场景图，从而支持开放词表分割（OVS）或指代表达分割（RES）等进一步的三维场景理解任务。与以往方法不同，我们的方法免训练，且不引入来自 CLIP/BERT 文本编码器的嵌入；相反，我们直接使用 MLLM 进行文本到文本的检索。通过大量实验，我们的方法相比近期研究展现出更优的性能，尤其是在复杂的指代表达分割（RES）任务上。
英文原文摘要 1019 字符
We propose OpenVoxel, a training-free algorithm for grouping and captioning sparse voxels for the open-vocabulary 3D scene understanding tasks. Given the sparse voxel rasterization (SVR) model obtained from multi-view images of a 3D scene, our OpenVoxel is able to produce meaningful groups that describe different objects in the scene. Also, by leveraging powerful Vision Language Models (VLMs) and Multi-modal Large Language Models (MLLMs), our OpenVoxel successfully build an informative scene map by captioning each group, enabling further 3D scene understanding tasks such as open-vocabulary segmentation (OVS) or referring expression segmentation (RES). Unlike previous methods, our method is training-free and does not introduce embeddings from a CLIP/BERT text encoder. Instead, we directly proceed with text-to-text search using MLLMs. Through extensive experiments, our method demonstrates superior performance compared to recent studies, particularly in complex referring expression segmentation (RES) tasks.