LifeEval:面向第一人称日常生活任务辅助 AI 的多模态基准
LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Tasks
为视觉研究者提供了一个以第一人称连续视频流为输入、要求模型在对话中实时给出任务导向协助的评测设定,可用于检验 MLLM 的在线感知与自适应交互能力,而非事后视频问答。
原文依据 egocentric real-time perception from continuous first-person streams, and human-assistant collaborative interaction through natural dialogues
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型(MLLMs)的快速进展标志着迈向通用人工智能的重要一步,为增强人类能力提供了巨大潜力。然而,它们在动态真实环境中提供有效协助的能力仍 largely 未被充分探索。现有视频基准主要通过回溯式分析或孤立的感知任务来评估被动理解,未能刻画实时用户协助所要求的交互性与自适应特性。为弥补这一空白,我们提出 LifeEval,一个多模态基准,旨在从第一人称视角评估日常生活场景中实时、任务导向的人机协作。LifeEval 强调三个关键方面:任务导向的整体性评估、基于连续第一人称流的第一人称实时感知,以及通过自然对话进行的人与助手协作交互。该基准通过严格的标注流程构建,包含覆盖 6 个核心能力维度的 4,075 个高质量问答对。在 LifeEval 上对 26 个最优 MLLM 的广泛评估揭示了在实现及时、有效且自适应交互方面存在显著挑战,为推进以人为中心的交互智能指明了关键方向。
英文原文摘要 1311 字符
The rapid progress of Multimodal Large Language Models (MLLMs) marks a significant step toward artificial general intelligence, offering great potential for augmenting human capabilities. However, their ability to provide effective assistance in dynamic, real-world environments remains largely underexplored. Existing video benchmarks predominantly assess passive understanding through retrospective analysis or isolated perception tasks, failing to capture the interactive and adaptive nature of real-time user assistance. To bridge this gap, we introduce LifeEval, a multimodal benchmark designed to evaluate real-time, task-oriented human-AI collaboration in daily life from an egocentric perspective. LifeEval emphasizes three key aspects: task-oriented holistic evaluation, egocentric real-time perception from continuous first-person streams, and human-assistant collaborative interaction through natural dialogues. Constructed via a rigorous annotation pipeline, the benchmark comprises 4,075 high-quality question-answer pairs across 6 core capability dimensions. Extensive evaluations of 26 state-of-the-art MLLMs on LifeEval reveal substantial challenges in achieving timely, effective and adaptive interaction, highlighting essential directions for advancing human-centered interactive intelligence.