Act2See：面向视频推理的涌现式主动视觉感知
Act2See: Emergent Active Visual Perception for Video Reasoning
把「取帧/生成帧」作为可学习的动作嵌入文本思维链，让 VLM 在推理过程中自主决定何时检索真实帧、何时合成反事实帧；这一「主动视觉感知」范式可直接迁移到需要多步视觉证据获取的视频问答与长视频理解任务。
原文依据 enables active visual perception by empowering VLMs to actively interleave video frames within text CoTs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）通常依赖静态的初始帧进行视频推理，限制了其在推理过程演进时纳入关键动态信息的能力。现有通过为思维链（CoT）补充额外帧信息的方法往往思维链质量欠佳，且缺乏为假设性或反事实场景合成视觉信息的关键能力。本文提出 Act-to-See（Act2See），一种新颖的框架，通过赋予 VLM 在文本思维链中主动穿插视频帧的能力来实现主动视觉感知。Act2See 基于由前沿 VLM 生成的高质量推理轨迹数据集进行监督微调（SFT）而构建。这些轨迹整合了对检索已有帧或生成新帧的主动调用，并经过严格校验，与人工标注的思维链对齐以确保质量。该方法培养出一种涌现能力：在推理时，模型主动决定何时搜索或合成所需的视觉证据。Act2See 在包括 VideoEspresso 与 ViTIB 在内的挑战性基准上取得了新的最优结果，并在 Video-MME、EgoNormia 与 VCR-Bench 上优于同等规模或更大规模的模型，展示了在赋予 VLM 主动视觉感知以进行视频推理方面的进展。代码：https://github.com/martinmamql/act2see。
英文原文摘要 1389 字符
Vision-Language Models (VLMs) typically rely on static initial frames for video reasoning, restricting their ability to incorporate essential dynamic information as the reasoning process evolves. Existing methods that augment Chain-of-Thought (CoT) with additional frame information often exhibit suboptimal CoT quality and lack the crucial ability to synthesize visual information for hypothetical or counterfactual scenarios. We introduce Act-to-See (Act2See), a novel framework that enables active visual perception by empowering VLMs to actively interleave video frames within text CoTs. Act2See is developed via Supervised Fine-Tuning (SFT) on a high-quality dataset of reasoning traces generated by a frontier VLM. These traces integrate active calls to either retrieve existing frames or generate new ones, and are rigorously verified against human-annotated CoTs to ensure quality. This approach cultivates an emergent capability: at inference time, the model actively determines when to search for or synthesize the necessary visual evidence. Act2See establishes new state-of-the-art results on challenging benchmarks, including VideoEspresso and ViTIB, and outperforms comparable or larger models on Video-MME, EgoNormia, and VCR-Bench, demonstrating an advancement in enabling VLMs with active visual perception for video reasoning. Code: https://github.com/martinmamql/act2see.