超越全局分数：以细粒度词元定位作为 LVLM 幻觉的鲁棒检测器
Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations
把「对象词元是否强定位到某个图像块」作为判据，用 patch 级注意力弥散度与语义对齐度两个统计特征做词元级幻觉检测，可作为一种无需额外训练、可解释的 VLM 输出校验插件，接入任意 LVLM 的解码过程。
原文依据 a faithful object token must be strongly grounded in a specific image region
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（LVLM）在视觉推理任务上表现强劲，但仍极易产生幻觉。现有检测方法主要依赖粗粒度的、整图层面的度量，来衡量对象词元与输入图像之间的关系。这种全局策略存在局限：幻觉词元可能在许多局部区域上呈现微弱但广泛分散的相关性，这些相关性聚合起来会形成具有欺骗性的高整体相关性，从而规避现有的全局幻觉检测器。我们从一个简单却关键的观察出发：一个忠实的对象词元必须强定位到某个特定的图像区域。基于这一洞见，我们提出一个图像块级幻觉检测框架，考察跨模型层的细粒度词元级交互。我们的分析揭示了幻觉词元的两个特征性signature：（i）它们产生弥散、非局部化的注意力模式，与紧凑、聚焦良好的注意力形成对比；（ii）它们无法与任何视觉区域表现出有意义的语义对齐。在这些发现的指导下，我们开发了一种轻量且可解释的检测方法，利用图像块级统计特征并结合隐藏层表征。我们的方法在词元级幻觉检测上达到最高 90% 的准确率，证明了细粒度结构分析在检测幻觉方面的优越性。
英文原文摘要 1423 字符
Large vision-language models (LVLMs) achieve strong performance on visual reasoning tasks but remain highly susceptible to hallucination. Existing detection methods predominantly rely on coarse, whole-image measures of how an object token relates to the input image. This global strategy is limited: hallucinated tokens may exhibit weak but widely scattered correlations across many local regions, which aggregate into deceptively high overall relevance, thus evading the current global hallucination detectors. We begin with a simple yet critical observation: a faithful object token must be strongly grounded in a specific image region. Building on this insight, we introduce a patch-level hallucination detection framework that examines fine-grained token-level interactions across model layers. Our analysis uncovers two characteristic signatures of hallucinated tokens: (i) they yield diffuse, non-localized attention patterns, in contrast to the compact, well-focused attention and (ii) they fail to exhibit meaningful semantic alignment with any visual region. Guided by these findings, we develop a lightweight and interpretable detection method that leverages patch-level statistical features, combined with hidden-layer representations. Our approach achieves up to 90% accuracy in token-level hallucination detection, demonstrating the superiority of fine-grained structural analysis for detecting hallucinations.