先定位后稀疏化：归因引导的稀疏策略用于视觉幻觉缓解
Locate-then-Sparsify: Attribution Guided Sparse Strategy for Visual Hallucination Mitigation
提出基于因果干预的逐层归因方法，量化各层与幻觉的相关性，并据此稀疏化特征引导强度；该思路可迁移到其他需要定位关键层或模块的视觉-语言任务中，如视觉定位、图像描述生成中的错误分析。
原文依据 we introduce an attribution method based on causal interventions to quantify the hallucination relevance of each layer
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管大型视觉-语言模型（LVLMs）取得了显著进展，但其产生幻觉的倾向削弱了可靠性并限制了更广泛的实际部署。在幻觉缓解方法中，特征引导作为一种有前景的方法，能够在不增加推理成本的情况下减少LVLMs的错误输出。然而，当前方法对所有层采用统一的特征引导。这种启发式策略忽略了层间差异，可能干扰与幻觉无关的层，最终导致通用任务性能下降。本文提出先定位后稀疏化特征引导（LTS-FS），这是一个即插即用的框架，根据每层的幻觉相关性控制引导强度。我们首先构建了一个包含词级和句子级幻觉案例的数据集。基于该数据集，我们引入了一种基于因果干预的归因方法，以量化每层的幻觉相关性。利用跨层的归因分数，我们提出了一种逐层策略，将这些分数转换为各层的特征引导强度，从而能够更精确地调整与幻觉相关的层。在多个LVLMs和基准上的大量实验表明，LTS-FS有效缓解了幻觉，同时保持了强劲的性能。代码可在 https://github.com/huttersadan/LTS-FS 获取。
英文原文摘要 1504 字符
Despite the significant advancements in Large Vision-Language Models (LVLMs), their tendency to generate hallucinations undermines reliability and restricts broader practical deployment. Among the hallucination mitigation methods, feature steering emerges as a promising approach that reduces erroneous outputs in LVLMs without increasing inference costs. However, current methods apply uniform feature steering across all layers. This heuristic strategy ignores inter-layer differences, potentially disrupting layers unrelated to hallucinations and ultimately leading to performance degradation on general tasks. In this paper, we propose Locate-Then-Sparsify for Feature Steering (LTS-FS), a plug-and-play framework which controls the steering intensity according to the hallucination relevance of each layer. We first construct a dataset comprising token-level and sentence-level hallucination cases. Based on this dataset, we introduce an attribution method based on causal interventions to quantify the hallucination relevance of each layer. With the attribution scores across layers, we propose a layerwise strategy that converts these scores into feature steering intensities for individual layers, enabling more precise adjustments specifically on hallucination-relevant layers. Extensive experiments across multiple LVLMs and benchmarks demonstrate that LTS-FS effectively mitigates hallucination while preserving strong performance. Codes are available at https://github.com/huttersadan/LTS-FS.