面向多模态思维链的理由增强解码
Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought
提供了一种推理时即插即用的解码策略：将图像条件与思维链理由条件的下一词元分布相乘，可直接套用到任意 LVLM 的多模态推理流程中，无需训练即可提升理由忠实度与答案准确率。
原文依据 we propose rationale-enhanced decoding (RED), a novel plug-and-play inference-time decoding strategy
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（LVLM）通过将预训练视觉编码器与大语言模型（LLM）相结合，展现出卓越的能力。与单模态 LLM 类似，思维链（CoT）提示已被引入 LVLM，通过基于视觉与文本输入生成中间理由来增强多模态推理。尽管人们普遍认为 CoT 能提升 LVLM 的视觉定位与准确率，我们的实验却揭示了一个关键挑战：现有 LVLM 在 CoT 推理中常常忽略所生成理由的内容。为解决该问题，我们将多模态 CoT 推理重新表述为以理由条件对数似然为核心的 KL 约束奖励最大化问题。作为其最优解，我们提出理由增强解码（RED），一种新颖的即插即用推理时解码策略。RED 通过将分别以图像为条件和以理由为条件的两个下一词元分布相乘，来协调视觉信息与理由信息。大量实验表明，在多个基准和多种 LVLM 上，RED 相较标准 CoT 及其他解码方法均能持续且显著地提升推理表现。本工作提供了一种实用且有效的方法，可同时提升 LVLM 中 CoT 推理的忠实度与准确率，为构建更可靠的、以理由为根基的多模态系统铺平了道路。
英文原文摘要 1324 字符
Large vision-language models (LVLMs) have demonstrated remarkable capabilities by integrating pre-trained vision encoders with large language models (LLMs). Similar to single-modal LLMs, chain-of-thought (CoT) prompting has been adapted for LVLMs to enhance multi-modal reasoning by generating intermediate rationales based on visual and textual inputs. While CoT is assumed to improve grounding and accuracy in LVLMs, our experiments reveal a key challenge: existing LVLMs often ignore the contents of generated rationales in CoT reasoning. To address this, we re-formulate multi-modal CoT reasoning as a KL-constrained reward maximization focused on rationale-conditional log-likelihood. As the optimal solution, we propose rationale-enhanced decoding (RED), a novel plug-and-play inference-time decoding strategy. RED harmonizes visual and rationale information by multiplying distinct image-conditional and rationale-conditional next token distributions. Extensive experiments show that RED consistently and significantly improves reasoning over standard CoT and other decoding methods across multiple benchmarks and LVLMs. Our work offers a practical and effective approach to improve both the faithfulness and accuracy of CoT reasoning in LVLMs, paving the way for more reliable rationale-grounded multi-modal systems.