视觉语言模型 · CVPR2026 ·2026.06.03

StaR-KVQA:面向隐式知识视觉问答的结构化推理链

StaR-KVQA: Structured Reasoning Traces for Implicit-Knowledge Visual Question Answering

Zhihao Wen、Wenkang Wei、Yuan Fang、Xingtong Yu、Hui Zhang、Weicheng Zhu 等 7 人

与你有关的部分

对做VQA/多模态推理的人,可复用其「符号关系路径+路径锚定自然语言解释」的双路推理链构造与离线数据筛选流程,把答案监督升级为结构感知的自蒸馏,无需外部检索器即可在单次自回归推理中提升可解释性与准确率。

原文依据 dual-path structured reasoning traces--symbolic relation paths over text and vision together with path-grounded natural-language explanations
可迁移到 知识型视觉问答多模态推理视觉问答可解释性

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

知识型视觉问答(KVQA)要求模型将图像中的实体与事实知识关联并进行推理。近期工作提出了其隐式知识变体 IK-KVQA,其中多模态大语言模型(MLLM)是唯一的知识来源,答案在不进行外部检索的情况下生成。然而,现有的 IK-KVQA 方法通常仅以答案作为监督进行训练:推理过程保持隐式,解释往往薄弱或不一致,且标准监督微调(SFT)后的泛化可能较为脆弱。我们提出 StaR-KVQA,一个为 IK-KVQA 配备双路结构化推理链的框架——文本与视觉上的符号关系路径,以及锚定于该路径的自然语言解释——从而提供比通用的仅答案监督更强的归纳偏置。这些推理链充当模态感知的脚手架,引导模型关注相关实体与属性,比通用的思维链监督提供更多结构,同时不将推理限制于任何单一固定路径。StaR-KVQA 使用单一开源 MLLM 构建并筛选推理链,以构建离线的推理链增强数据集,随后执行结构感知的自蒸馏;不使用外部检索器、验证器或人工整理的知识库,推理为单次自回归前向。在多个基准上,StaR-KVQA 一致地提升了答案准确率与中间推理的透明度,在 OK-VQA 上相较最强基线取得最高 +11.3% 的答案准确率提升。

英文原文摘要 1570 字符

Knowledge-based Visual Question Answering (KVQA) requires models to ground entities in images and reason over factual knowledge. Recent work has introduced its implicit-knowledge variant, IK-KVQA, where a multimodal large language model (MLLM) is the sole knowledge source and answers are produced without external retrieval. Existing IK-KVQA approaches, however, are typically trained with answer-only supervision: reasoning remains implicit, justifications are often weak or inconsistent, and generalization after standard supervised fine-tuning (SFT) can be brittle. We propose StaR-KVQA, a framework that equips IK-KVQA with dual-path structured reasoning traces--symbolic relation paths over text and vision together with path-grounded natural-language explanations--to provide a stronger inductive bias than generic answer-only supervision. These traces act as modality-aware scaffolds that guide the model toward relevant entities and attributes, offering more structure than generic chain-of-thought supervision while not constraining reasoning to any single fixed path. With a single open-source MLLM, StaR-KVQA constructs and selects traces to build an offline trace-enriched dataset and then performs structure-aware self-distillation; no external retrievers, verifiers, or curated knowledge bases are used, and inference is a single autoregressive pass. Across benchmarks, StaR-KVQA consistently improves both answer accuracy and the transparency of intermediate reasoning, achieving up to +11.3% higher answer accuracy on OK-VQA over the strongest baseline.

标签

会议 CVPR2026
页码 5114-5124
Bibkey
来源 CVF Open Access