GeoPredict:利用预测式运动学与 3D 高斯几何实现精确的 VLA 操作
GeoPredict: Leveraging Predictive Kinematics and 3D Gaussian Geometry for Precise VLA Manipulation
把 3D 高斯几何预测与关键点轨迹预测作为训练期监督(通过深度渲染),推理时只加少量 query token、不做 3D 解码——这套「重监督、轻推理」的几何先验注入方式可直接迁移到任何连续动作 VLA 策略上,用来补足其 2D 反应式的空间推理短板。
原文依据 These predictive modules serve exclusively as training-time supervision through depth-based rendering, while inference requires only lightweight additional query tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言动作(VLA)模型在机器人操作中展现出很强的泛化能力,但总体上仍是反应式的、以 2D 为中心,因此在需要精确 3D 推理的任务中并不可靠。我们提出 GeoPredict,一个几何感知的 VLA 框架,为连续动作策略引入预测式运动学先验与几何先验。GeoPredict 包含一个轨迹级模块,用于编码运动历史并预测机械臂多步 3D 关键点轨迹;还包含一个预测式 3D 高斯几何模块,沿未来关键点轨迹以轨迹引导的细化方式预测工作空间几何。这些预测模块仅通过基于深度的渲染作为训练期监督,而推理时只需轻量的额外查询 token,无需调用任何 3D 解码。在 RoboCasa Human-50、LIBERO 以及真实世界操作任务上的实验表明,GeoPredict 稳定优于强 VLA 基线,在几何密集与空间要求高的场景中尤为明显。
英文原文摘要 1042 字符
Vision-Language-Action (VLA) models achieve strong generalization in robotic manipulation but remain largely reactive and 2D-centric, making them unreliable in tasks that require precise 3D reasoning. We propose GeoPredict, a geometry-aware VLA framework that augments a continuous-action policy with predictive kinematic and geometric priors. GeoPredict introduces a trajectory-level module that encodes motion history and predicts multi-step 3D keypoint trajectories of robot arms, and a predictive 3D Gaussian geometry module that forecasts workspace geometry with track-guided refinement along future keypoint trajectories. These predictive modules serve exclusively as training-time supervision through depth-based rendering, while inference requires only lightweight additional query tokens without invoking any 3D decoding. Experiments on RoboCasa Human-50, LIBERO, and real-world manipulation tasks show that GeoPredict consistently outperforms strong VLA baselines, especially in geometry-intensive and spatially demanding scenarios.