PAS：用于检测大型视觉语言模型中物体幻觉的预生成注意力分数
PAS: Prelim Attention Score for Detecting Object Hallucinations in Large Vision-Language Models
PAS 提供了一种无需额外前向传播、仅利用解码时对已生成 token 的注意力权重来实时检测物体幻觉的信号，可直接用于 LVLM 推理时的过滤与干预，也为分析 LVLM 的图像依赖程度提供了可复用的量化手段。
原文依据 we introduce the Prelim Attention Score (PAS), a lightweight, training-free signal computed from attention weights over prelim tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（LVLM）能力强大，但由于物体幻觉的存在，它们仍然不可靠。在本工作中，我们表明在许多幻觉性预测中，LVLM 实际上忽略了图像，转而依赖先前生成的输出（“预生成”）token 来推断新物体。我们通过以预生成为条件计算图像与预测物体之间的互信息来量化这一行为，证明微弱的图像依赖与幻觉强烈相关。基于这一发现，我们引入了预生成注意力分数（PAS），这是一种轻量级、免训练的信号，由对预生成 token 的注意力权重计算得到。PAS 无需额外的前向传播，可在推理过程中即时计算。利用这一此前被忽视的信号，PAS 在多个模型和数据集上实现了最优的物体幻觉检测性能，支持实时过滤与干预。
英文原文摘要 944 字符
Large vision-language models (LVLMs) are powerful, yet they remain unreliable due to object hallucinations. In this work, we show that in many hallucinatory predictions the LVLM effectively ignores the image and instead relies on previously generated output ("prelim") tokens to infer new objects. We quantify this behavior via the mutual information between the image and the predicted object conditioned on the prelim, demonstrating that weak image dependence strongly correlates with hallucination. Building on this finding, we introduce the Prelim Attention Score (PAS), a lightweight, training-free signal computed from attention weights over prelim tokens. PAS requires no additional forward passes and can be computed on the fly during inference. Exploiting this previously overlooked signal, PAS achieves state-of-the-art object-hallucination detection across multiple models and datasets, enabling real-time filtering and intervention.