CausalLens：敏感度引导的多头因果干预用于大型视觉语言模型幻觉缓解
CausalLens: Sensitivity-Guided Multi-Head Causal Intervention for Hallucination Mitigation in Large Vision-Language Models
提出一种免训练、单次前向的推理时干预：按视觉、文本、系统提示词三条通路分解注意力头，用敏感度度量挑出视觉可靠的注意力头并调整其中层隐状态贡献，再以投影对齐校正稳定多头融合后的状态。做 LVLM 幻觉缓解或视觉 grounding 的研究者可直接复用这套注意力头级敏感度分析与隐状态校正流程，替代多遍/对比解码以降低延迟。
原文依据 identifies visually reliable heads using a sensitivity measure and selectively adjusts their mid-layer hidden-state contributions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期的大型视觉语言模型（LVLM）在多模态理解与生成方面展现出令人印象深刻的能力。尽管取得了这些进展，它们仍然容易出现幻觉，即由于过度依赖文本先验，模型输出与视觉输入相冲突。现有的推理时缓解方法往往依赖多遍或对比解码，这会增加延迟并限制其在实时场景中的适用性。为解决这一局限，我们提出 CausalLens，一种免训练、单次前向的干预方法，直接调整解码器隐状态以增强视觉 grounding。通过将注意力头分解为视觉、文本与系统提示词三条通路，CausalLens 利用敏感度度量识别出视觉可靠的注意力头，并有选择地调整其中层隐状态的贡献。一种投影对齐的校正进一步在多头融合之后稳定这些被调整的状态，确保增强后的视觉信息在整个解码过程中得以保留。在多个幻觉基准与多种 LVLM 架构上的大量实验表明，CausalLens 在带来可忽略的计算开销的同时，持续提升视觉保真度。该方法无需微调或架构改动，因而非常适合对延迟敏感的实际应用。
英文原文摘要 1369 字符
Recent Large Vision-Language Models (LVLMs) have shown impressive capabilities in multimodal understanding and generation. Despite this progress, they remain prone to *hallucination*, where model outputs conflict with the visual input due to an over-reliance on textual priors. Existing inference-time mitigation approaches frequently depend on multi-pass or contrastive decoding, which increases latency and limits their applicability in real-time settings. To address this limitation, we propose **CausalLens**, a training-free and single-pass intervention that directly adjusts the decoder hidden states to strengthen visual grounding. By decomposing attention heads into visual, textual, and system prompt pathways, CausalLens identifies visually reliable heads using a sensitivity measure and selectively adjusts their mid-layer hidden-state contributions. A projection-aligned correction further stabilizes these adjusted states after multi-head fusion, ensuring that the enhanced visual information is preserved throughout decoding. Extensive experiments across multiple hallucination benchmarks and LVLM architectures demonstrate that CausalLens consistently improves visual fidelity while adding negligible computational overhead. The method requires no fine-tuning or architectural changes, making it well-suited for practical, latency-sensitive applications.