ConsisVLA-4D：面向机器人操作，在高效三维感知与四维推理中推进时空一致性
ConsisVLA-4D: Advancing Spatiotemporal Consistency in Efficient 3D-Perception and 4D-Reasoning for Robotic Manipulation
对做视觉研究的人，这篇提供了不依赖额外深度传感器、仅从多视角 2D 观测构建紧凑潜变量几何 token 的思路（CO-Fuser），以及用跨视角物体身份对齐来保证语义一致性的做法（CV-Aligner），可直接迁移到需要多视角一致表征的 VLA 或具身操作策略中。
原文依据 eliminating spatial relation ambiguities between objects across views using compact latent representations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前的视觉-语言-动作（VLA）模型主要关注将二维观测映射为动作，但在时空感知与推理方面存在明显局限：1）空间表征往往依赖额外传感器，带来可观的计算开销；2）视觉推理通常局限于未来帧预测，缺乏与基于指令的场景之间的对齐，从而损害了时空一致性。为应对这些挑战，我们提出 ConsisVLA-4D，一个统一且高效的框架，用于增强三维感知与四维推理中的时空一致性。具体而言，我们设计了：1）CV-Aligner，通过筛选与指令相关的区域并在多个视角间对齐物体身份，确保跨视角的物体语义一致性；2）CO-Fuser，利用紧凑的潜变量表征消除跨视角物体之间的空间关系歧义，从而保证跨物体的空间几何一致性。在此基础上，我们引入 3）CS-Thinker，以在动作展开过程中实现跨场景的时空一致性。它从 CV-Aligner 的物体语义 token 中学习局部动力学的隐式知识，并从 CO-Fuser 的几何 token 中学习全局深度，从而增强场景变化下的高效视觉推理。大量实验表明，得益于其高效的时空一致性设计，ConsisVLA-4D 在 LIBERO 基准和真实世界平台上相比 OpenVLA 分别取得了 21.6% 和 41.5% 的性能提升，以及 2.3 倍和 2.4 倍的推理加速。
英文原文摘要 1678 字符
Current Vision-Language-Action (VLA) models primarily focus on mapping 2D observations to actions but exhibit notable limitations in spatiotemporal perception and reasoning: 1) spatial representations often rely on additional sensors, introducing substantial computational overhead; 2) visual reasoning is typically limited to future-frame prediction, lacking alignment with the instruction-grounded scene and thus compromising spatiotemporal consistency. To address these challenges, we propose **ConsisVLA-4D**, a unified and efficient framework that enhances spatiotemporal consistency in 3D-Perception and 4D-Reasoning. Specifically, we design: **1) CV-Aligner**, which ensures **C**ross-**V**iew object semantic consistency via filtering instruction-relevant regions and aligning object identities across multiple viewpoints; **2) CO-Fuser**, which guarantees **C**ross-**O**bject spatial geometric consistency by eliminating spatial relation ambiguities between objects across views using compact latent representations. Building upon these, we introduce **3) CS-Thinker** to achieve **C**ross-**S**cene spatiotemporal consistency as actions unfold. It learns implicit knowledge of local dynamics from object-semantic tokens of CV-Aligner and global depth from geometric tokens of CO-Fuser, thereby enhancing efficient visual reasoning under scene variations. Extensive experiments demonstrate that, benefiting from its efficient spatiotemporal consistency design, ConsisVLA-4D achieves **21.6%** and **41.5%** performance improvements, along with **2.3x** and **2.4x** inference speedups compared to OpenVLA on the LIBERO benchmark and real-world platforms, respectively.