R-C2：循环一致性强化学习提升多模态推理
R-C2: Cycle-Consistent Reinforcement Learning Improves Multimodal Reasoning
用跨模态循环一致性构造无标签的密集奖励，可直接迁移到 VLM 的视觉问答与多模态推理训练中，替代投票式自一致性来抑制模态偏置。
原文依据 cross-modal inconsistency provides a rich, natural signal for learning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论鲁棒的感知与推理要求各感知模态之间保持一致。然而，当前的多模态模型常常违背这一原则，对同一输入的视觉表示与文本表示给出相互矛盾的预测。我们没有用标准投票机制来掩盖这些失败（投票机制会放大系统性偏差），而是证明跨模态不一致性为学习提供了丰富而自然的信号。我们提出 R-C2，一个通过强制跨模态循环一致性来解决内部冲突的强化学习框架。该框架要求模型执行反向推理、切换模态，并通过前向推理可靠地重建答案，从而建立一个密集的、无需标签的奖励。这一循环约束迫使模型自主对齐其表征。针对该结构进行优化可缓解模态特定错误，并将推理准确率提升最多 7.6 个百分点。我们的结果表明，高级推理能力不仅来自数据规模的扩大，也来自对世界强加一种结构上一致的理解。
英文原文摘要 1086 字符
Robust perception and reasoning require consistency across sensory modalities. Yet, current multimodal models often violate this principle, yielding contradictory predictions for visual versus textual representations of the same input. Rather than masking these failures with standard voting mechanisms--which amplify systematic biases--we demonstrate that cross-modal inconsistency provides a rich, natural signal for learning. We introduce R-C2, a reinforcement learning framework that resolves internal conflicts by enforcing cross-modal cycle consistency. By requiring a model to perform backward inference, switches modalities, and reliably reconstruct the answer via forward inference, we establish a dense, label-free reward. This cyclic constraint forces the model to autonomously align its representations. Optimizing for this structure mitigates modality-specific errors and improves reasoning accuracy by up to 7.6 points. Our results suggest that advanced reasoning emerges not just from scaling data, but from enforcing a structurally consistent understanding of the world.