在 360 度中思考：野外场景下的人形视觉搜索
Thinking in 360deg: Humanoid Visual Search in the Wild
把 360° 全景图当作可主动转头的沉浸式环境，构建了 MLLM 智能体做物体/路径搜索的基准与后训练方案；对做 VLM 空间推理、全景/具身视觉的人，可直接复用其 H* Bench 评测设定与后训练提升范式。
原文依据 a humanoid agent actively rotates its head to search for objects or paths in an immersive world represented by a 360deg panoramic image
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类依赖头部（头动）与眼睛（眼动）的协同控制，在 360 度环境中高效搜索视觉信息。然而，以往的视觉搜索方法局限于静态图像，忽略了物理具身性及其与三维世界的交互。我们如何构建与人类一样高效的具身视觉搜索智能体，同时绕开真实世界硬件带来的限制？为此，我们提出人形视觉搜索：人形智能体主动转动头部，在以 360 度全景图表示的沉浸式世界中搜索物体或路径。为研究视觉拥挤的真实场景中的视觉搜索，我们构建了 H* Bench，这是一个新的基准，它超越了家庭场景，转向需要高级视觉-空间推理能力的具有挑战性的野外场景，例如交通枢纽、大型零售空间、城市街道和公共机构。我们的实验首先揭示，即便是顶级的闭源模型也表现不佳，在物体搜索和路径搜索上仅取得 30% 的成功率。随后我们采用后训练技术增强开源模型，在最小的 3B 模型上，物体搜索（14.83% - 47.38%）和路径搜索（6.44% - 24.94%）的成功率均提升三倍以上。值得注意的是，路径搜索较低的上限揭示了其固有难度，我们将其归因于对复杂空间常识的需求。我们的结果不仅展示了一条有前景的前进路径，也量化了在构建能够无缝融入人类日常生活的 MLLM 智能体方面仍然存在的巨大挑战。
英文原文摘要 1624 字符
Humans rely on the synergistic control of head (cephalomotor) and eye (oculomotor) to efficiently search for visual information in 360deg. However, prior approaches to visual search are limited to a static image, neglecting the physical embodiment and its interaction with the 3D world. How can we develop embodied visual search agents as efficient as humans while bypassing the constraints imposed by real-world hardware? To this end, we propose humanoid visual search where a humanoid agent actively rotates its head to search for objects or paths in an immersive world represented by a 360deg panoramic image. To study visual search in visually-crowded real-world scenarios, we build H* Bench, a new benchmark that moves beyond household scenes to challenging in-the-wild scenes that necessitate advanced visual-spatial reasoning capabilities, such as transportation hubs, large-scale retail spaces, urban streets, and public institutions. Our experiments first reveal that even top-tier proprietary models falter, achieving only 30% success in object and path search. We then use post-training techniques to enhance the open-source models, increasing its success rate by over threefold for both object search (14.83% - 47.38%) and path search (6.44% - 24.94%) on the smallest 3B model. Notably, the lower ceiling of path search reveals its inherent difficulty, which we attribute to the demand for sophisticated spatial commonsense. Our results not only show a promising path forward but also quantify the immense challenge that remains in building MLLM agents that can be seamlessly integrated into everyday human life.