MindPower：在基于视觉语言模型的具身智能体中实现心智理论推理
MindPower: Enabling Theory-of-Mind Reasoning in VLM-based Embodied Agents
把心智理论（ToM）推理显式嵌入 VLM 具身智能体的感知-推理-决策-行动流水线，并用 Mind-Reward 优化目标对齐推理与行为，为视觉研究者提供了一个可复用的「以机器人为中心」的推理监督与奖励设计范式。
原文依据 we introduce Mind-Reward, a novel optimization objective that encourages VLMs to produce consistent ToM Reasoning and behavior
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论心智理论（Theory of Mind, ToM）指推断他人心理状态（如信念、欲望与意图）的能力。当前基于视觉语言模型的具身智能体缺乏基于心智理论的决策能力，而现有基准仅关注人类的心理状态，忽略了智能体自身的视角，从而阻碍了连贯的决策与动作生成。为解决这一问题，我们提出 MindPower，一个以机器人为中心、融合感知、心理推理、决策与行动的框架。给定多模态输入，MindPower 首先感知环境与人类状态，随后执行心智理论推理以对自我与他人进行建模，最后在推断出的心理状态引导下生成决策与动作。此外，我们引入 Mind-Reward，一种新的优化目标，促使视觉语言模型产生一致的心智理论推理与行为。我们的模型在决策上比 GPT-4o 高出 12.77%，在动作生成上高出 12.49%。项目主页：https://zhangdaxia22.github.io/MindPower/。
英文原文摘要 989 字符
Theory of Mind (ToM) refers to the ability to infer others' mental states, such as beliefs, desires, and intentions. Current vision-language embodied agents lack ToM-based decision-making, and existing benchmarks focus solely on human mental states while ignoring the agent's own perspective, hindering coherent decision and action generation. To address this, we propose MindPower, a Robot-Centric framework integrating Perception, Mental Reasoning, Decision Making and Action. Given multimodal inputs, MindPower first perceives the environment and human states, then performs ToM Reasoning to model both self and others, and finally generates decisions and actions guided by inferred mental states. Furthermore, we introduce Mind-Reward, a novel optimization objective that encourages VLMs to produce consistent ToM Reasoning and behavior. Our model outperforms GPT-4o by 12.77% in decision making and 12.49% in action generation. Project page: https://zhangdaxia22.github.io/MindPower/.