APPO：面向视频推理的注意力引导感知策略优化
APPO: Attention-guided Perception Policy Optimization for Video Reasoning
把强化学习中的组内相对优势换成基于注意力、跨响应聚焦同一关键帧的 token 级稠密奖励，可在无细粒度标注下提升视频多模态模型的感知能力，这套奖励设计可直接迁移到其他视频 VLM 的 RL 后训练流程。
原文依据 leverages token-level dense rewards to improve model's fine-grained perception
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论实际上，复杂视频推理过度依赖细粒度感知，而非专家级（如博士、科学级）推理能力。通过大量实证观察，我们认识到感知的关键影响。具体而言，当感知能力几乎固定时，将推理能力从 Qwen3-8B 提升到 OpenAI-o3 仅带来 0.7% 的性能提升；相反，即使感知模型规模仅有微小变化（从 7B 到 32B），也能带来 1.4% 的性能提升，这表明提升感知而非推理对性能改进更为关键。因此，探索如何通过推理增强感知能力而无需昂贵的细粒度标注信息是值得的。为实现这一目标，我们专门提出 APPO，即注意力引导感知策略优化算法，利用 token 级稠密奖励来提升模型的细粒度感知。APPO 的核心思想是优化来自不同响应、但主要关注同一关键视频帧的那些 token（称为组内感知 token）。在不同视频基准和不同规模模型（3B/7B）上的实验结果表明，APPO 持续优于 GRPO 和 DAPO（0.5%~4%）。我们希望本工作提供一种有前景的方法，以低成本方式通过推理有效增强模型的感知能力，服务于多样化的场景与需求。
英文原文摘要 1432 字符
Complex video reasoning, actually, relies excessively on fine-grained perception rather than on expert (e.g., Ph.D, Science)-level reasoning.Through extensive empirical observation, we have recognized the critical impact of perception.In particular, when perception ability is almost fixed, enhancing reasoning from Qwen3-8B to OpenAI-o3 yields only 0.7% performance improvement. Conversely, even minimal change in perception model scale (from 7B to 32B) boosts performance by 1.4%, indicating enhancing perception, rather than reasoning, is more critical to improve performance.Therefore, exploring how to enhance perception ability through reasoning without the need for expensive fine-grained annotation information is worthwhile.To achieve this goal, we specially propose APPO, the Attention-guided Perception Policy Optimization algorithm that leverages token-level dense rewards to improve model's fine-grained perception.The core idea behind APPO is to optimize those tokens from different responses that primarily focus on the same crucial video frame (called intra-group perception tokens).Experimental results on diverse video benchmarks and models with different scales (3/7B) demonstrate APPO consistently outperforms GRPO and DAPO (0.5% 4%). We hope our work provides a promising approach to effectively enhance model's perception abilities through reasoning in a low-cost manner, serving diverse scenarios and demands.