打破幻觉：多模态解码中正负路径的相遇
Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding
PND 是一种无需训练的推理阶段解码干预：用多层注意力放大显著视觉证据作为正路径，同时削弱核心物体特征构造反事实负路径，逐步对比两者输出以压制语言先验主导的幻觉。做 VLM 幻觉缓解或解码策略改进的工作可直接复用该双路径对比框架，且它揭示了视觉特征在注意力中被系统性低估这一可进一步探查的现象。
原文依据 a training-free inference framework that intervenes directly in the decoding process to enforce visual fidelity
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）常因过度依赖语言先验而受到物体幻觉的损害，即生成与视觉现实相矛盾的内容。我们提出正负解码（Positive-and-Negative Decoding, PND），这是一个无需训练的推理框架，直接干预解码过程以强化视觉保真度。PND 的动机来自我们的关键发现：VLM 中存在严重的注意力缺陷，视觉特征在经验上被赋予过低的权重。我们的框架通过双路径对比来纠正这一问题：正路径利用多层注意力放大显著的视觉证据，以鼓励忠实的描述，从而直接抵消注意力缺陷；同时，负路径识别并削弱核心物体的特征，以构造强反事实，从而惩罚缺乏视觉依据、由先验主导的生成。通过在每一步对比模型在这两种视角下的输出，PND 将生成引导向不仅在语言上可能、而且在视觉上符合事实的文本。在 POPE、MME 和 CHAIR 等基准上的大量实验表明，PND 取得了最优性能，准确率最高提升 6.5%，在显著减少物体幻觉的同时还增强了描述细节，且完全无需重新训练模型。该方法可有效泛化到 LLaVA、InstructBLIP、InternVL 和 Qwen-VL 等多种 VLM 架构。
英文原文摘要 1443 字符
Vision-Language Models (VLMs) are frequently undermined by object hallucination--generating content that contradicts visual reality--due to an over-reliance on linguistic priors. We introduce Positive-and-Negative Decoding (PND), a training-free inference framework that intervenes directly in the decoding process to enforce visual fidelity. PND is motivated by our key finding of a critical attention deficit in VLMs, where visual features are empirically under-weighted. Our framework corrects this via a dual-path contrast: The positive path amplifies salient visual evidence using multi-layer attention to encourage faithful descriptions, directly counteracting the attention deficit. Simultaneously, the negative path identifies and degrades the core object's features to create a strong counterfactual, which penalizes ungrounded, prior-dominant generation. By contrasting the model's outputs from these two perspectives at each step, PND steers generation towards text that is not just linguistically probable, but visually factual. Extensive experiments on benchmarks like POPE, MME, and CHAIR show that PND achieves state-of-the-art performance with up to 6.5% accuracy improvement, substantially reducing object hallucination while also enhancing descriptive detail--all without requiring any model retraining. The method generalizes effectively across diverse VLM architectures including LLaVA, InstructBLIP, InternVL, and Qwen-VL.