在不确定性中思考：利用潜变量熵感知解码缓解多模态大模型中的幻觉
Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decoding
提出一种基于熵感知的解码策略，通过概率加权连续嵌入和视觉锚框注入来缓解多模态大模型在视觉问答中的幻觉，可作为一种即插即用的推理时增强手段用于视觉语言模型。
原文依据 we propose Latent Entropy-Aware Decoding (LEAD), an efficient plug-and-play decoding strategy that leverages semantic context to achieve reliable reasoning.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期多模态大推理模型（MLRMs）的进展显著提升了视觉问答的性能。然而，我们观察到过渡词（如because、however和wait）与幻觉密切相关，且往往表现出高熵状态。我们认为，充分的上下文推理信息可以直接从词元概率分布中提取。受叠加表示理论启发，我们提出利用潜变量叠加推理来整合多个候选语义并维持潜变量推理轨迹。假设是，依赖离散文本输入可能驱使模型走向顺序显式推理，在高熵推理阶段未能充分利用密集的上下文线索。因此，我们提出从词元概率分布构建丰富的语义表示以增强上下文推理。为此，我们提出潜变量熵感知解码（LEAD），一种利用语义上下文实现可靠推理的高效即插即用解码策略。我们方法的核心在于熵感知推理模式切换。模型在高熵状态下采用概率加权的连续嵌入，并随着熵降低切换回离散词元嵌入。此外，我们提出一种先验引导的视觉锚框注入策略，鼓励模型关注视觉信息。大量实验表明，LEAD在多个基准上有效缓解了各种MLRMs中的幻觉。
英文原文摘要 1581 字符
Recent advancements in multimodal large reasoning models (MLRMs) have significantly improved performance in visual question answering. However, we observe that transition words (e.g., because, however, and wait) are closely associated with hallucinations and tend to exhibit high-entropy states. We argue that adequate contextual reasoning information can be directly extracted from the token probability distribution. Inspired by superposed representation theory, we propose leveraging latent superposed reasoning to integrate multiple candidate semantics and maintain latent reasoning trajectories. The hypothesis is that reliance on discrete textual inputs may drive the model toward sequential explicit reasoning, underutilizing dense contextual cues during high-entropy reasoning stages. Therefore, we propose constructing rich semantic representations from the token probability distributions to enhance in-context reasoning. With this goal, we present Latent Entropy-Aware Decoding (LEAD), an efficient plug-and-play decoding strategy that leverages semantic context to achieve reliable reasoning. The heart of our method lies in entropy-aware reasoning mode switching. The model employs probability-weighted continuous embeddings under high-entropy states and transitions back to discrete token embeddings as entropy decreases. Moreover, we propose a prior-guided visual anchor injection strategy that encourages the model to focus on visual information. Extensive experiments show that LEAD effectively mitigates hallucinations across various MLRMs on multiple benchmarks.