视觉语言模型 · CVPR2026 ·2026.06.03

强化学习以学习,选举以推理:视频推理的双范式

Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning

Songyuan Yang、Weijiang Yu、Jilin Ma、Ziyu Liu、Guijian Tang、Wenjing Yang 等 8 人

与你有关的部分

提出一种训练时用强化学习生成结构化证据、推理时用证据加权选举的范式,可迁移到视频问答、时序定位等需要证据对齐的视觉语言任务,提升推理可靠性与可解释性。

原文依据 We introduce Reinforce to Learn, Elect to Reason (RLER), a dual paradigm that decouples learning to produce evidence from obtaining a reliable answer.
可迁移到 视频问答时序定位视频推理

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

视频推理随着大型多模态模型(LMM)的发展而进步,然而其推理往往是一次前向传播就返回答案,而不验证推理是否与证据对齐。我们提出强化学习以学习、选举以推理(RLER),这是一种双范式,将学习产生证据与获得可靠答案解耦。在RLER-Training中,我们用组相对强化学习(RL)和3种新颖的任务驱动奖励来优化策略:帧敏感奖励将推理建立在显式关键帧上,思考透明奖励塑造可读且可解析的推理轨迹,抗重复奖励提升信息密度。这些信号教会模型输出结构化、机器可检查的证据,并增强推理能力。在RLER-Inference中,我们应用一个免训练编排器,生成一小组多样候选,解析其答案和引用的帧,根据证据一致性、置信度、透明度和非冗余性对它们评分,然后执行稳健的证据加权选举。这闭合了产生和使用证据之间的循环,在不增大模型的情况下提高了可靠性和可解释性。我们在8个代表性基准上全面评估RLER,与各种开源和基于RL的LMM进行比较。RLER在所有基准上达到最先进水平,相比基础模型平均提升6.3%,同时每个问题平均使用3.1个候选,表明计算与质量之间取得了良好平衡。结果支持一个简单的论点:在学习时使证据显式化,在推理时按证据选举,是通往可信视频推理的稳健路径。

英文原文摘要 1706 字符

Video reasoning has advanced with large multimodal models (LMMs), yet their inference is often a single pass that returns an answer without verifying whether the reasoning is evidence-aligned. We introduce **Reinforce to Learn, Elect to Reason (RLER)**, a dual paradigm that decouples learning to produce evidence from obtaining a reliable answer. In **RLER-Training**, we optimize the policy with group-relative reinforcement learning (RL) and 3 novel task-driven rewards: Frame-sensitive reward grounds reasoning on explicit key frames, Think-transparency reward shapes readable and parsable reasoning traces, and Anti-repetition reward boosts information density. These signals teach the model to emit structured, machine-checkable evidence and potentiate reasoning capabilities. In **RLER-Inference**, we apply a train-free orchestrator that generates a small set of diverse candidates, parses their answers and cited frames, scores them by evidence consistency, confidence, transparency, and non-redundancy, and then performs a robust evidence-weighted election. This closes the loop between producing and using evidence, improving reliability and interpretability without enlarging the model. We comprehensively evaluate RLER against various open-source and RL-based LMMs on 8 representative benchmarks. RLER achieves state of the art across all benchmarks and delivers an average improvement of 6.3 % over base models, while using on average 3.1 candidates per question, indicating a favorable balance between compute and quality. The results support a simple thesis: making evidence explicit during learning and electing by evidence during inference is a robust path to trustworthy video reasoning.

标签

会议 CVPR2026
页码 33794-33804
Bibkey
来源 CVF Open Access