可扩展的物体关系编码：提升大语言模型中的三维空间推理能力
Scalable Object Relation Encoding for Better 3D Spatial Reasoning in Large Language Models
QuatRoPE 用四元数式整体向量编码 3D 坐标，并通过注意力层点积隐式算出成对空间关系，使输入长度随物体数线性增长；做 3D 场景-语言对齐或把场景图喂给 LLM 的工作可直接复用这一位置编码与 IGRE 门控扩展思路。
原文依据 we propose QuatRoPE, a novel positional embedding method with an input length that is linear to the number of objects
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论空间推理关注在三维场景中依据空间关系定位目标物体，对构建智能具身智能体至关重要。由于三维场景-语言配对数据有限，从零训练具备强推理能力的模型十分困难。已有方法尝试将三维场景表示注入大语言模型（LLM）的输入空间，并利用其预训练的理解与推理能力完成空间推理。然而，编码绝对位置的模型难以从过早融合的特征中提取空间关系，而显式编码所有空间关系（数量随物体数呈二次增长）作为输入 token 的方法则面临可扩展性差的问题。为解决上述局限，我们提出 QuatRoPE，一种输入长度随物体数线性增长的新型位置嵌入方法，它通过注意力层中的点积显式计算成对空间关系。QuatRoPE 对三维坐标的整体向量编码保证了高度的空间一致性，维持了对场景几何完整性的忠实。此外，我们引入隔离门控 RoPE 扩展（IGRE），有效将 QuatRoPE 的影响限制在与物体相关的 token 上，从而最小化对 LLM 既有位置嵌入的干扰，并保持 LLM 的原有能力。大量实验验证了所提方法的有效性。代码与数据见 https://github.com/oceanflowlab/QuatRoPE。
英文原文摘要 1605 字符
Spatial reasoning focuses on locating target objects based on spatial relations in 3D scenes, which plays a crucial role in developing intelligent embodied agents. Due to the limited availability of 3D scene-language paired data, it is challenging to train models with strong reasoning ability from scratch. Previous approaches have attempted to inject 3D scene representations into the input space of Large Language Models (LLMs) and leverage the pretrained comprehension and reasoning abilities for spatial reasoning. However, models encoding absolute positions struggle to extract spatial relations from prematurely fused features, while methods explicitly encoding all spatial relations (which is quadratic in the number of objects) as input tokens suffer from poor scalability. To address these limitations, we propose QuatRoPE, a novel positional embedding method with an input length that is linear to the number of objects, and explicitly calculates pairwise spatial relations through the dot product in attention layers. QuatRoPE's holistic vector encoding of 3D coordinates guarantees a high degree of spatial consistency, maintaining fidelity to the scene's geometric integrity. Additionally, we introduce the Isolated Gated RoPE Extension (IGRE), which effectively limits QuatRoPE's influence to object-related tokens, thereby minimizing interference with the LLM's existing positional embeddings and maintaining the LLM's original capabilities. Extensive experiments demonstrate the effectiveness of our approaches. The code and data are available at https://github.com/oceanflowlab/QuatRoPE.