HulluEdit:单次前向的证据一致子空间编辑用于缓解大型视觉语言模型中的幻觉
HulluEdit: Single-Pass Evidence-Consistent Subspace Editing for Mitigating Hallucinations in Large Vision-Language Models
提出一种无需参考模型、单次前向的正交子空间编辑方法,将隐藏状态分解为视觉证据、冲突先验和残差不确定性三个正交子空间,仅对先验子空间进行编辑以抑制幻觉,同时数学上保证视觉成分不受影响。该技术可直接用于提升现有LVLM的可靠性,且推理高效。
原文依据 we decompose the hidden states of the model into orthogonal subspaces--visual evidence, conflicting priors, and residual uncertainty
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型(LVLM)中的物体幻觉严重阻碍了其可靠部署。现有方法难以平衡效率与准确性:它们通常需要昂贵的参考模型和多次前向传播,或应用静态编辑而存在抑制真实视觉证据的风险。为解决这一问题,我们提出了HulluEdit,一种单次前向、无需参考模型的干预框架。我们的核心创新是正交子空间编辑:我们将模型的隐藏状态分解为正交子空间——视觉证据、冲突先验和残差不确定性——从而能够选择性地抑制幻觉模式而不干扰视觉 grounding。该方法在数学上保证了对先验子空间施加的编辑完全不影响视觉成分。大量实验表明,HulluEdit在包括POPE和CHAIR在内的基准上,跨多种架构实现了最优的幻觉减少,同时在MME上保持了通用能力并维持了高效推理。我们的方法始终优于对比解码和静态子空间编辑基线,为更可信的LVLM提供了新途径。代码已发布于 https://github.com/VioAgnes/HulluEdit。
英文原文摘要 1277 字符
Object hallucination in Large Vision-Language Models (LVLMs) significantly hinders their reliable deployment. Existing methods struggle to balance efficiency and accuracy: they often require expensive reference models and multiple forward passes, or apply static edits that risk suppressing genuine visual evidence. To address this, we introduce HulluEdit, a single-pass, reference-free intervention framework. Our core innovation is orthogonal subspace editing: we decompose the hidden states of the model into orthogonal subspaces--visual evidence, conflicting priors, and residual uncertainty--enabling selective suppression of hallucinatory patterns without interfering with visual grounding. This approach mathematically guarantees that edits applied to the prior subspace leave the visual component entirely unaffected. Extensive experiments show that HulluEdit achieves state-of-the-art hallucination reduction on benchmarks including POPE and CHAIR across diverse architectures, while preserving general capabilities on MME and maintaining efficient inference. Our method consistently outperforms contrastive decoding and static subspace editing baselines, offering a new pathway toward more trustworthy LVLMs. Code is released at https://github.com/VioAgnes/HulluEdit.