VES-RFT：以视觉证据敏感性为奖励缓解大型视觉语言模型中的幻觉
VES-RFT: Rewarding Visual Evidence Sensitivity to Mitigate Hallucinations in Large Vision-Language Models
把「有图/无图两次前向的熵差」定义成可计算的视觉证据敏感度奖励，配合自动核验物体正确性的可验证奖励，用无 critic 的 GRPO 做训练时对齐；做 VLM 幻觉或视觉 grounding 的工作可以直接复用这套双奖励与并行双前向的训练范式，且推理仍是单次前向、无额外开销。
原文依据 we measure decision uncertainty with and without the image, and define a Visual Evidence Sensitivity (VES) signal as the image-attributable change in entropy
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）即使在提供图像的情况下也常常过度依赖语言先验，从而导致物体幻觉。本文从视觉证据如何塑造模型不确定性的角度重新审视物体级幻觉。对每个输入，我们分别测量有图像和无图像时的决策不确定性，并将图像所引起的信息熵变化定义为视觉证据敏感性（VES）信号。基于该信号，我们提出视觉证据敏感性强化微调（VES-RFT），这是一种在训练阶段显式奖励模型依赖正确视觉证据的强化微调方法。我们将这一连续、无需标注的信号与一个可验证奖励相结合，后者通过自动将生成的物体提及与图像进行核对来强制物体事实正确性，从而得到一个无需人工标注即可计算的目标函数。我们使用带 KL 正则化的无 critic GRPO 优化该双重目标，训练时只需并行的有图与无图前向，同时保持推理阶段的单次前向。在多个 VLM 系列与基准上，VES-RFT 一致地抑制物体幻觉，并提升模型在歧义情形下的鲁棒性。具体而言，在 LLaVA-7B 上，VES-RFT 使 MS-COCO 的 CHAIR_S/CHAIR_I 分别下降 12.8 和 1.8，并使 POPE 准确率提升 4.92%。大量实验表明，将不确定性转化为可学习的奖励，并与可验证的正确性信号相结合，为训练阶段的幻觉缓解与更强的视觉 grounding 提供了一种可扩展的机制。
英文原文摘要 1554 字符
Vision-Language Models (VLMs) often over-rely on linguistic priors even when images are provided, leading to object hallucinations. We revisit object-wise hallucination from the perspective of how visual evidence shapes the model's uncertainty. For each input, we measure decision uncertainty with and without the image, and define a Visual Evidence Sensitivity (VES) signal as the image-attributable change in entropy. Building on this signal, we introduce Visual Evidence Sensitivity Reinforcement Fine-Tuning (VES-RFT), a training-time reinforcement fine-tuning method that explicitly rewards reliance on correct visual evidence. We pair this continuous, annotation-free signal with a verifiable reward that enforces factual object correctness by automatically checking generated object mentions against the image, yielding a computable objective without human annotations. We optimize the dual objective using critic-free GRPO with KL regularization, requiring only parallel image and no-image passes during training while preserving single-pass inference. Across multiple VLM families and benchmarks, VES-RFT consistently suppresses object hallucinations and improves robustness under ambiguity. Specifically, on LLaVA-7B, VES-RFT reduces 12.8 and 1.8 on CHAIR_S/CHAIR_I of MS-COCO, and increases POPE accuracy by 4.92%. Extensive experiments indicate that turning uncertainty into a learnable reward, paired with verifiable correctness signals, provides a scalable mechanism for training-time hallucination mitigation and stronger visual grounding.