SoPE：基于球坐标的位置嵌入用于增强3D大视觉语言模型的空间感知
SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs
针对3D点云词元的位置编码缺陷，提出球坐标位置嵌入（SoPE），可替换RoPE用于3D LVLMs，提升空间感知与方向变化建模，为视觉语言模型中的3D几何表示提供新思路。
原文依据 we introduce Spherical Coordinate-based Positional Embedding (SoPE). Our method maps point-cloud token indices into a 3D spherical coordinate space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于大语言模型（LLM）构建的3D大视觉语言模型（3D LVLMs）在各种多模态任务中取得了显著进展。然而，其继承的位置相关建模机制——旋转位置嵌入（RoPE）——对于3D多模态理解而言仍非最优。原始RoPE公式在编码3D词元时无法保留必要的三维空间结构，其相对距离计算忽略了角度依赖性，阻碍了模型捕捉视觉表示中方向变化的能力。为克服这些局限，我们引入了基于球坐标的位置嵌入（SoPE）。该方法将点云词元索引映射到3D球坐标空间，实现了空间位置与方向角度的统一建模。这一公式保留了点云数据固有的几何结构，增强了空间感知，并为多模态学习产生了更一致且更具表现力的几何表示。此外，我们引入了一种多尺度频率混合策略，以融合不同频率域的特征信息。在多个3D场景基准上的实验结果验证了该方法的有效性，而真实世界部署实验进一步展示了其强大的泛化能力。
英文原文摘要 1356 字符
3D Large Vision-Language Models (3D LVLMs) built upon Large Language Models (LLMs) have achieved remarkable progress across various multimodal tasks. However, their inherited position-dependent modeling mechanism, Rotary Position Embedding (RoPE), remains suboptimal for 3D multimodal understanding. The vanilla RoPE formulation fails to preserve essential three-dimensional spatial structures when encoding 3D tokens, and its relative distance computation overlooks angular dependencies hindering the model's ability to capture directional variations in visual representations. To overcome these limitations, we introduce Spherical Coordinate-based Positional Embedding (SoPE). Our method maps point-cloud token indices into a 3D spherical coordinate space, enabling unified modeling of spatial locations and directional angles. This formulation preserves the inherent geometric structure of point-cloud data, enhances spatial awareness, and yields more consistent and expressive geometric representations for multimodal learning. In addition, we introduce a multi-scale frequency mixing strategy to fuse feature information across different frequency domains. Experimental results on multiple 3D scene benchmarks validate the effectiveness of our approach, while real-world deployment experiments further demonstrate its strong generalization capability.