Conan:在多尺度视觉证据上像侦探一样推理的渐进式学习
Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidence
把「帧级证据定位」显式建模为推理轨迹的一部分(识别-推理-动作),并用 RLVR 分阶段冷启动训练,可直接迁移到需要视觉 grounding 的多步 VQA、长视频问答与视频 agent 决策场景。
原文依据 Conan identifies context and evidence frames, reasons over cross-frame clues, and adaptively decides when to conclude or explore further.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频推理需要在帧之间进行多步演绎,仍是多模态大语言模型(MLLM)面临的重大挑战。基于强化学习(RL)的方法虽然提升了推理能力,但往往依赖纯文本的推理链,导致结论缺乏依据或产生幻觉。相反,帧检索类方法引入了视觉 grounding,却仍受困于证据定位不准确。为解决这些局限,本文提出 Conan,一个以证据为依托的多步视频推理框架。Conan 识别上下文帧与证据帧,在跨帧线索上进行推理,并自适应地决定何时结束推理或继续探索。为此,本文 1)构建了 Conan-91K,一个大规模数据集,包含自动生成的推理轨迹,涵盖帧识别、证据推理与动作决策;2)设计了多阶段渐进式冷启动策略,并结合识别-推理-动作(AIR)RLVR 训练框架,以渐进方式激励多步视觉推理。在六个多步推理基准上的大量实验表明,Conan 相比基线 Qwen2.5-VL-7B-Instruct 平均准确率提升超过 10%,达到最优性能。此外,Conan 能有效泛化到长视频理解任务,验证了其强大的可扩展性与鲁棒性。
英文原文摘要 1407 字符
Video reasoning, which requires multi-step deduction across frames, remains a major challenge for multimodal large language models (MLLMs). While reinforcement learning (RL)-based methods enhance reasoning capabilities, they often rely on text-only chains that yield ungrounded or hallucinated conclusions. Conversely, frame-retrieval approaches introduce visual grounding, yet still struggle with inaccurate evidence localization. To address these limitations, we present Conan, a framework for evidence-grounded multi-step video reasoning. Conan identifies context and evidence frames, reasons over cross-frame clues, and adaptively decides when to conclude or explore further. To achieve this, we **1)** construct Conan-91K, a large-scale dataset of automatically generated reasoning traces that include frame identification, evidence reasoning, and action decision, and **2)** design a multi-stage progressive cold-start strategy combined with an Identification-Reasoning-Action (AIR) RLVR training framework to progressively incentivize multi-step visual reasoning. Extensive experiments on six multi-step reasoning benchmarks demonstrate that Conan surpasses the baseline Qwen2.5-VL-7B-Instruct by an average of over 10% in accuracy, achieving state-of-the-art performance. Furthermore, Conan generalizes effectively to long video understanding tasks, validating its strong scalability and robustness.