从跨视图关系学习多视图空间推理
Learning Multi-View Spatial Reasoning from Cross-View Relations
XVR 提供了跨视图对应、关系验证与目标定位三类 VQA 监督信号，可直接用于微调 VLM 以增强多视图空间表征；其数据构造方式（从 3D 场景与机器人轨迹自动生成问答）也可复用于构建其他具身空间推理训练集。
原文依据 XVR comprises 100K vision-question-answer samples derived from 18K diverse 3D scenes and 70K robotic manipulation trajectories
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）在单视图视觉任务上已取得令人瞩目的成果，但缺乏具身智能系统理解三维环境、跨不同视角操作物体所必需的多视图空间推理能力。在本工作中，我们提出跨视图关系（Cross-View Relations，XVR），这是一个大规模数据集，旨在教会 VLM 跨多个视图进行空间推理。XVR 包含 10 万个视觉问答样本，源自 1.8 万个多样化的三维场景和 7 万条机器人操作轨迹，涵盖三项基础空间推理任务：对应（跨视图匹配物体）、验证（校验空间关系）与定位（识别物体位置）。在 XVR 上微调的 VLM 在现有多视图与机器人空间推理基准（MindCube 与 RoboSpatial）上取得显著提升。当作为视觉-语言-动作模型的骨干网络集成时，经 XVR 训练得到的表征提升了 RoboCasa 上的成功率。我们的结果表明，对跨视图空间关系进行显式训练能够显著增强多视图推理能力，并有效迁移至真实世界的机器人操作。
英文原文摘要 1191 字符
Vision-language models (VLMs) have achieved impressive results on single-view vision tasks, but lack the multi-view spatial reasoning capabilities essential for embodied AI systems to understand 3D environments and manipulate objects across different viewpoints. In this work, we introduce Cross-View Relations (XVR), a large-scale dataset designed to teach VLMs spatial reasoning across multiple views. XVR comprises 100K vision-question-answer samples derived from 18K diverse 3D scenes and 70K robotic manipulation trajectories, spanning three fundamental spatial reasoning tasks: Correspondence (matching objects across views), Verification (validating spatial relationships), and Localization (identifying object positions). VLMs fine-tuned on XVR achieve substantial improvements on established multi-view and robotic spatial reasoning benchmarks (MindCube and RoboSpatial). When integrated as backbones in Vision-Language-Action models, XVR-trained representations improve success rates on RoboCasa. Our results demonstrate that explicit training on cross-view spatial relations significantly enhances multi-view reasoning and transfers effectively to real-world robotic manipulation.