平板包装基准：通过家具组装评估大型视觉语言模型的时空理解
Flat-Pack Bench: Evaluating Spatio-Temporal Understanding in Large Vision-Language Models through Furniture Assembly
该基准通过家具组装任务和视觉提示，系统评测LVLM的细粒度时序排序、状态定位、部件匹配与跟踪能力，为视频理解模型在复杂空间交互场景下的评估提供了新协议和诊断工具。
原文依据 we introduce Flat-Pack Bench, a novel benchmark centered on furniture assembly tasks
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（LVLM）的出现显著提升了视频理解能力。然而，现有基准主要关注粗粒度任务，如动作分割、分类、描述和检索。此外，这些基准通常依赖于能够通过语言轻松识别的实体，如家居物品、动物、人物等，限制了它们在复杂、真实世界视频场景中的适用性。但是，许多应用如家具组装、烹饪等，需要对视频进行逐步的细粒度时空理解，而当前基准对此评估不足。为填补这一空白，我们引入了Flat-Pack Bench，一个以家具组装任务为核心的新基准。我们的基准通过多项选择题并辅以突出相关部件的视觉提示作为细粒度问题的参考，评估LVLM在细微任务上的表现，包括组装动作的时序排序、组装状态的时序定位、理解部件匹配以及跟踪。实验表明，最优的LVLM在细粒度时空推理方面存在显著困难，突显了它们在有效利用视频时序信息、有限的跟踪能力以及理解物理接触等空间交互方面的局限性。
英文原文摘要 1395 字符
The emergence of Large Vision-Language Models (LVLMs) has significantly advanced video understanding capabilities. However, existing benchmarks focus predominantly on coarse-grained tasks such as action segmentation, classification, captioning, and retrieval. Furthermore, these benchmarks often rely on entities that can be easily identified verbally, like household objects, animals, human subjects, etc., limiting their applicability to complex, in-the-wild video scenarios. But, many applications such as furniture assembly, cooking, etc., require step-by-step fine-grained spatio-temporal understanding of the video, which is not sufficiently evaluated in current benchmarks.To address this gap, we introduce Flat-Pack Bench, a novel benchmark centered on furniture assembly tasks. Our benchmark evaluates LVLMs on nuanced tasks, including temporal ordering of assembly actions, temporal localization of assembly state, understanding part mating, and tracking, using multiple-choice questions paired with visual prompts highlighting relevant parts as references for fine-grained questions. Our experiments reveal that state-of-the-art LVLMs struggle significantly with fine-grained spatio-temporal reasoning, highlighting their limitations in effectively leveraging temporal information from videos, limited tracking ability, and understanding of spatial interactions like physical contact.