CC-VQA：面向知识型视觉问答中知识冲突缓解的冲突与相关性感知方法
CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering
对做知识型 VQA 的人，这篇给出了一个免训练、可直接插到现有 VLM 推理流程里的方案：先用视觉语义做内外知识冲突判定，再用相关性加权的位置编码压缩与自适应解码抑制低相关检索段落，可作为检索增强 VLM 推理阶段的通用后处理模块复用。
原文依据 we propose CC-VQA: a novel training-free, conflict- and correlation-aware method for KB-VQA
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论知识型视觉问答（KB-VQA）在处理知识密集型任务方面展现出显著潜力。然而，由于预训练所得的模型知识是静态的，视觉语言模型（VLM）中的静态参数知识与动态检索到的信息之间会产生冲突。模型输出要么忽略检索到的上下文，要么与参数知识呈现出不一致的融合，这给 KB-VQA 带来了实质性挑战。现有的知识冲突缓解方法大多改编自基于语言的方法，通过精心设计的提示策略或上下文感知的解码机制来处理上下文层面的冲突。然而，这些方法忽视了视觉信息在冲突中的关键作用，并且受制于冗余的检索上下文，从而损害了冲突识别的准确性与缓解的有效性。为解决上述局限，我们提出 CC-VQA：一种面向 KB-VQA 的新型免训练、冲突与相关性感知的方法。该方法包含两个核心组件：（1）以视觉为中心的上下文冲突推理，在内部与外部知识上下文之间进行视觉—语义冲突分析；（2）相关性引导的编码与解码，其中包含针对低相关性陈述的位置编码压缩，以及使用相关性加权冲突评分的自适应解码。在 E-VQA、InfoSeek 与 OK-VQA 基准上的大量评测表明，CC-VQA 取得了最优性能，相较现有方法取得 3.3% 至 6.4% 的绝对准确率提升。代码将在论文录用后发布。
英文原文摘要 1627 字符
Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks. However, conflicts arise between static parametric knowledge in vision language models (VLMs) and dynamically retrieved information due to the static model knowledge from pre-training. The outputs either ignore retrieved contexts or exhibit inconsistent integration with parametric knowledge, posing substantial challenges for KB-VQA. Current knowledge conflict mitigation methods primarily adapted from language-based approaches, focusing on context-level conflicts through engineered prompting strategies or context-aware decoding mechanisms. However, these methods neglect the critical role of visual information in conflicts and suffer from redundant retrieved contexts, which impair accurate conflict identification and effective mitigation. To address these limitations, we propose CC-VQA: a novel training-free, conflict- and correlation-aware method for KB-VQA. Our method comprises two core components: (1) Vision-Centric Contextual Conflict Reasoning, which performs visual-semantic conflict analysis across internal and external knowledge contexts; and (2) Correlation-Guided Encoding and Decoding, featuring positional encoding compression for low-correlation statements and adaptive decoding using correlation-weighted conflict scoring. Extensive evaluations on E-VQA, InfoSeek, and OK-VQA benchmarks demonstrate that CC-VQA achieves state-of-the-art performance, yielding absolute accuracy improvements of 3.3% to 6.4% compared to existing methods. Code will be released upon acceptance.