HanDyVQA：面向细粒度手物交互动态的视频问答基准
HanDyVQA: A Video QA Benchmark for Fine-Grained Hand-Object Interaction Dynamics
提供了一个细粒度手物交互视频问答基准，并配套物体/部件级分割掩码，可用于评测视频基础模型的时空与部件级几何推理能力；其发现「显式注入 HOI 线索到视觉特征可提升性能」为设计 HOI 感知的视频表征提供了可复用思路。
原文依据 We also found that incorporating explicit HOI cues into visual features improves performance
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论手物交互（HOI）涉及人类操作对物体产生时空效应的动态过程。然而，现有的语义 HOI 基准要么只关注操作，要么只关注粗粒度的效应，缺乏捕捉 HOI 动态所需的细粒度时空推理。我们提出 HanDyVQA，一个细粒度视频问答基准，全面覆盖 HOI 的操作与效应两个方面。HanDyVQA 包含六种互补的问题类型（动作、过程、物体、位置、状态变化和物体部件），共计 11.1K 道多项选择问答对。所收集的问答对要求识别操作风格、手/物体运动以及部件级状态变化。HanDyVQA 还包含 10.3K 个针对物体与物体部件的分割掩码，从而能够评估视频目标分割中的物体/部件级推理。我们在该基准上评测了近期视频基础模型，发现即使表现最好的 Gemini-2.5-Pro 也仅达到 73% 的准确率，远低于人类表现（97%）。进一步分析揭示了在空间关系、运动以及部件级几何理解方面仍然存在的挑战。我们还发现，将显式 HOI 线索融入视觉特征可提升性能，为未来 HOI 感知模型提供了启示。
英文原文摘要 1296 字符
Hand-object interaction (HOI) involves dynamics where human manipulations produce spatio-temporal effects on objects. However, existing semantic HOI benchmarks focus on either manipulation or effects at a coarse level, lacking fine-grained spatio-temporal reasoning to capture HOI dynamics. We introduce HanDyVQA, a fine-grained video QA benchmark that comprehensively covers both the manipulation and effect aspects of HOI. HanDyVQA comprises six complementary question types (Action, Process, Objects, Location, State Change, and Object Parts), totaling 11.1K multiple-choice QA pairs. Collected QA pairs require recognizing manipulation styles, hand/object motions, and part-level state changes. HanDyVQA also includes 10.3K segmentation masks for Objects and Object Parts, enabling the evaluation of object/part-level reasoning in video object segmentation. We evaluated recent video foundation models on our benchmark and found that even the best, Gemini-2.5-Pro, achieved only 73% accuracy, well below human performance (97%). Further analysis shows the remaining challenges in spatial relationships, motion, and part-level geometric understanding. We also found that incorporating explicit HOI cues into visual features improves performance, providing insights for future HOI-aware models.