INSIGHT基准：面向机器人操作的场景内引导视觉定位
INSIGHT Bench: Towards Grounded IN-SItu Guidance for Robotic ManipulaTion
论文提出的“in-situ guide grounding”任务要求视觉语言动作模型（VLA）理解物体上的文字和符号（如“PULL”），这为视觉定位研究提供了新的场景和评测基准；其可扩展的仿真框架和程序化生成方法可用于训练和评估模型对物体中心信息的理解能力。
原文依据 we introduce INSIGHT Bench, a benchmark that formalizes the task of "in-situ guide grounding"
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类直觉上依赖物体上刻有的文字和符号（例如“PULL”、“Squeeze and Turn”）来安全正确地执行任务。相比之下，视觉-语言-动作模型擅长遵循外部语言指令，但在很大程度上仍无法感知这种以物体为中心的信息。这一能力对于可靠的机器人操作至关重要，然而由于缺乏标准化的基准，相关进展仍无法衡量。为了填补这一空白，我们推出了INSIGHT基准，该基准形式化了“场景内引导视觉定位”任务。INSIGHT基准提供了一个全面的分类体系，用于评估智能体如何利用多样化的引导信息，包括动作方向提示和程序性指令。它还包含一个可扩展的仿真框架，能够程序化生成任务，并以编程方式将每个视觉引导与其对应的物理约束关联起来。我们发布了该基准及由此产生的轨迹数据集，以支持未来研究。我们对近期VLA模型的评估揭示了一个关键局限：它们对场景内引导进行视觉定位的能力不一致，且强烈依赖于信息类型。虽然模型在某些引导类别上表现成功，但在其他类别上经常失败。然而，当相同信息以语言指令形式提供时，性能大幅提升，这表明如果VLA能够解读场景内引导，这些引导可能有助于提升操作性能。这些发现强调了进一步研究理解和视觉定位场景内引导的必要性。
英文原文摘要 1563 字符
Humans intuitively rely on text and symbols inscribed on objects (e.g. "PULL", "Squeeze and Turn") to perform tasks safely and correctly. In contrast, vision-language-action models excel at following external language commands, but remain largely unaware of this object-centric information. This capability is essential for reliable robotic operation, yet progress remains unmeasured due to the absence of standardized benchmarks. To address this gap, we introduce INSIGHT Bench, a benchmark that formalizes the task of "in-situ guide grounding". INSIGHT Bench provides a comprehensive taxonomy that evaluates how agents utilize diverse guide information, including action-direction cues and procedural instructions. It also includes a scalable simulation framework that procedurally generates tasks and programmatically links each visual guide to its corresponding physical constraint. We release both the benchmark and the resulting trajectory dataset to support future research. Our evaluation of recent VLA models reveals a critical limitation: their ability to ground in-situ guides is inconsistent and strongly dependent on the type of information. While models succeed on some guide categories, they frequently fail on others. However, performance improves substantially when the same information is provided as language instructions, indicating that in-situ guides could contribute to manipulation performance if VLAs were capable of interpreting them. These findings underscore the need for further research on understanding and grounding in-situ guides.