首词元 Logit 提升：缓解大型视觉语言模型中对象幻觉的视觉定位方法
First Logit Boosting: Visual Grounding Method to Mitigate Object Hallucination in Large Vision-Language Models
FLB 是一种免训练的解码策略，通过复用首个生成词元的 logit 来抑制 LVLM 中的对象幻觉，可即插即用地应用于任何自回归视觉语言模型，且推理开销可忽略。
原文依据 First Logit Boosting (FLB), a simple yet effective training-free technique designed to alleviate long-term decay in LVLMs.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期的大型视觉语言模型（LVLM）在需要理解视觉和语言输入的各种多模态任务中表现出显著性能。然而，对象幻觉——在答案中生成不存在的对象——仍然是一个持续存在的挑战。尽管已提出重训练和外部视觉定位等方法以缓解该问题，但它们仍受限于高数据成本或结构复杂性。诸如对比解码（CD）之类的免训练方法更具成本效益，避免了额外训练或外部模型，但仍受长期衰减困扰，即随着生成推进，视觉定位减弱且语言先验占据主导。本文提出首词元 Logit 提升（FLB），一种简单而有效的免训练技术，旨在缓解 LVLM 中的长期衰减。FLB 存储首个生成词元的 logit 并将其加到后续词元预测中，有效缓解视觉信息的长期衰减。我们观察到 FLB（1）在整个生成过程中维持嵌入首词元的视觉信息，且（2）通过“The”词元的稳定效应抑制幻觉词。实验结果表明，FLB 在多种任务、基准和骨干模型上显著减少对象幻觉。值得注意的是，它带来的推理开销可忽略不计，使其高度适用于实时多模态系统。代码见 https://github.com/jiwooha20/FLB。
英文原文摘要 1541 字符
Recent Large Vision-Language Models (LVLMs) have demonstrated remarkable performance across various multimodal tasks that require understanding both visual and linguistic inputs. However, object hallucination -- the generation of nonexistent objects in answers -- remains a persistent challenge. Although several approaches such as retraining and external grounding methods have been proposed to mitigate this issue, they still suffer from high data costs or structural complexity. Training-free methods such as Contrastive Decoding (CD) are more cost-effective, avoiding additional training or external models, but still suffer from long-term decay, where visual grounding weakens and language priors dominate as the generation progresses. In this paper, we propose First Logit Boosting (FLB), a simple yet effective training-free technique designed to alleviate long-term decay in LVLMs. FLB stores the logit of the first generated token and adds it to subsequent token predictions, effectively mitigating long-term decay of visual information. We observe that FLB (1) sustains the visual information embedded in the first token throughout generation, and (2) suppresses hallucinated words through the stabilizing effect of the "The" token. Experimental results show that FLB significantly reduces object hallucination across various tasks, benchmarks, and backbone models. Notably, it causes negligible inference overhead, making it highly applicable to real-time multimodal systems. Code is available at https://github. com/jiwooha20/FLB