BOP-ASK：面向视觉语言模型的物体交互推理
BOP-ASK: Object-Interaction Reasoning for Vision-Language Models
利用 BOP 数据集的 6D 物体位姿自动派生抓取位姿、指代物体位姿、路径轨迹与相对空间/深度关系，为 VLM 的空间与交互推理提供可规模化、带精确几何真值的训练与评测信号；这套「从位姿标注反推问答」的流水线可直接迁移到其他需要细粒度 3D 监督的视觉-语言任务。
原文依据 Our data generation pipeline leverages 6D object poses from the Benchmark for Object Pose Estimation (BOP) datasets
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）在空间推理基准上已取得令人印象深刻的表现，但这些评测掩盖了其在理解物体交互方面的关键弱点。现有基准测试的是高层关系（如「在……左边」「在……后面」等），却忽略了真实应用所需的细粒度空间理解：精确的三维定位、物体之间的物理兼容性、物体可供性以及多步空间规划。在本工作中，我们提出 BOP-ASK，一个用于物体交互推理的新型大规模数据集，同时服务于训练与基准评测。我们的数据生成流水线利用来自物体姿态估计基准（BOP）数据集的 6D 物体位姿，由此派生出抓取位姿、指代物体位姿、路径规划轨迹、相对空间与深度关系以及物体间关系等细粒度标注。BOP-ASK 包含超过 15 万张图像和 3300 万条问答对，覆盖六项任务（其中四项为全新任务），为训练与评测 VLM 提供了丰富资源。我们评测了闭源与开源 VLM，并在贡献的测试基准 BOP-ASK-core 上开展了人工评测。我们还发布了 BOP-ASK-lab，这是一个图像并非来自 BOP 的分布外基准，可用于测试泛化能力。实验表明，在 BOP-ASK 上训练的模型优于基线，并展现出精确的物体与抓取姿态估计、轨迹规划以及在杂乱环境中细粒度以物体为中心的空间推理等涌现能力。项目网站：https://bop-ask.github.io/
英文原文摘要 1623 字符
Vision-Language Models (VLMs) have achieved impressive performance on spatial reasoning benchmarks, yet these evaluations mask critical weaknesses in understanding object interactions. Current benchmarks test high-level relationships ("left of," "behind", etc.) but ignore fine-grained spatial understanding needed for real-world applications: precise 3D localization, physical compatibility between objects, object affordances and multi-step spatial planning. In this work, we present BOP-ASK, a novel large-scale dataset for object-interaction reasoning for both training and benchmarking. Our data generation pipeline leverages 6D object poses from the Benchmark for Object Pose Estimation (BOP) datasets from which we derive fine-grained annotations such as grasp poses, referred object poses, path planning trajectories, relative spatial and depth relationships, and object-to-object relationships. BOP-ASK comprises over 150k images and 33M question-answer pairs spanning six tasks (four novel), providing a rich resource for training and evaluating VLMs. We evaluate proprietary and open-sourced VLMs, and conduct human evaluations on BOP-ASK-core, a contributed test benchmark. We also release BOP-ASK-lab, an out-of-distribution benchmark with images not sourced from BOP, enabling testing of generalization. Our experiments demonstrate that models trained on BOP-ASK outperform baselines and exhibit emergent capabilities such as precise object and grasp pose estimation, trajectory planning, and fine-grained object-centric spatial reasoning in cluttered environments. Project website: https://bop-ask.github.io/