LongVideo-R1:面向低成本长视频理解的智能导航
LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding
把长视频问答建模为「由粗到细的主动片段导航」:用层级视觉摘要作为推理起点,让 MLLM 智能体逐步定位信息量最大的片段,并在信息足够时提前停止,从而在低算力预算下完成长视频理解;其两阶段 SFT+RL 训练与奖励设计可直接迁移到其他需要选择性采样的视觉-语言任务。
原文依据 the agent initiates traversal from top-level visual summaries and iteratively refines its focus, immediately halting the exploration process upon acquiring sufficient knowledge
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文针对低计算预算下的长视频理解这一关键且尚未被充分探索的挑战展开研究。我们提出 LongVideo-R1,一种具备推理能力的主动式多模态大语言模型(MLLM)智能体,用于高效地导航视频上下文,避免穷举式搜索带来的冗余。LongVideo-R1 的核心是一个推理模块,它利用高层视觉线索推断出信息量最大的视频片段以供后续处理。在推理阶段,该智能体从顶层视觉摘要出发进行遍历,并迭代地细化其关注焦点;一旦获得足以回答查询的知识,便立即停止探索过程。为便于训练,我们首先从带有视觉定位标注的视频语料 CGBench 中提取层级式视频描述,并引导 GPT-5 生成 33K 条高质量的「思维链加工具调用」轨迹。LongVideo-R1 智能体基于 Qwen-3-8B 模型、通过两阶段范式进行微调:先监督微调(SFT),后强化学习(RL),其中 RL 采用专门设计的奖励函数,以最大化片段导航的选择性与高效性。在多个长视频基准上的实验验证了该方法的有效性,其在问答准确率与效率之间取得了更优的权衡。所有整理的数据与源代码均在补充材料中提供,并将公开发布。
英文原文摘要 1431 字符
This paper addresses the critical and underexplored challenge of long video understanding with low computational budgets.We propose LongVideo-R1, an active, reasoning-equipped multimodal large language model (MLLM) agent designed for efficient video context navigation, avoiding the redundancy of exhaustive search.At the core of LongVideo-R1 lies a reasoning module that leverages high-level visual cues to infer the most informative video clip for subsequent processing.During inference, the agent initiates traversal from top-level visual summaries and iteratively refines its focus, immediately halting the exploration process upon acquiring sufficient knowledge to answer the query.To facilitate training, we first extract hierarchical video captions from CGBench, a video corpus with grounding annotations, and guide GPT-5 to generate 33K high-quality chain-of-thought-with-tool trajectories. The LongVideo-R1 agent is fine-tuned upon the Qwen-3-8B model through a two-stage paradigm: supervised fine-tuning (SFT) followed by reinforcement learning (RL), where RL employs a specifically designed reward function to maximize selective and efficient clip navigation.Experiments on multiple long video benchmarks validate the effectiveness of name, which enjoys superior tradeoff between QA accuracy and efficiency. All curated data and source code are provided in the supplementary material and will be made publicly available.