先构想、再关注、后回应：大型视觉语言模型中的反事实幻觉缓解
Envision, Attend, Then Respond: Counterfactual Hallucination Mitigation in Large Vision-Language Models
提出一种免训练框架，利用扩散先验生成视觉印象并定位反事实元素，再通过对比解码抑制幻觉，可复用于提升LVLM在视觉证据与常识冲突时的鲁棒性。
原文依据 We propose Envision-Attend-Respond (EnAR), a training-free framework that leverages visual priors to steer the model's attention toward counterfactual elements in the image.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（LVLMs）在视觉证据与世界知识冲突时（即反事实场景）常产生幻觉。我们提出“构想-关注-回应”（EnAR）框架，这是一种免训练框架，利用视觉先验引导模型注意力转向图像中的反事实元素。构想阶段通过调用扩散先验进行潜变量扰动来构建视觉印象，生成输入图像的一个先验一致的对应版本。关注阶段通过LVLM的视觉编码器处理原始图像及其视觉印象，以定位反事实元素，形成相应的填充输入。回应阶段在原始输入与填充输入之间进行对比解码，以抑制偏差并增强视觉理解。实验表明，EnAR持续缓解幻觉并提升响应保真度，在VLMBias上获得10.82%的提升，在POPE上平均提升6.9%，在反事实和一般幻觉设置下均展现出鲁棒性。此外，该框架在异构LVLM架构上依然有效，为多模态推理中的幻觉治理提供了新视角。
英文原文摘要 1224 字符
Large Vision-Language Models (LVLMs) often hallucinate when visual evidence conflicts with world knowledge, i.e., in counterfactual scenarios. We propose Envision-Attend-Respond (EnAR), a training-free framework that leverages visual priors to steer the model's attention toward counterfactual elements in the image. The Envision stage constructs a visual impression by invoking a diffusion prior to perform latent perturbations, yielding a prior-consistent counterpart of the input image. The Attend stage processes the original image and its visual impression through the LVLM's vision encoder to localize counterfactual elements, forming a corresponding padded input. The Respond stage performs contrastive decoding between the original and padded inputs to suppress bias and enhance visual understanding. Empirically, EnAR consistently mitigates hallucinations and improves response fidelity, achieving a 10.82% gain on VLMBias and an average 6.9% improvement on POPE, demonstrating robustness across both counterfactual and general hallucination settings. Moreover, the framework remains effective across heterogeneous LVLM architectures, offering a new perspective for hallucination governance in multimodal reasoning.