VKG-QA：面向大型多模态模型的视觉知识图谱问答
VKG-QA: Visual Knowledge Graph-based Question Answer for Large Multimodal Models
把知识图谱渲染成图结构可视化图像，让 LMM 直接感知并推理高阶关系，为视觉推理研究提供了一个可复用的「图→图」表征范式与评测基准，可用于检验和提升模型对结构化视觉关系的理解能力。
原文依据 knowledge graphs are transformed into graphical visualizations that LMMs can directly perceive and reason over
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论对结构化知识进行理解与推理是智能系统的一项基础能力。大型语言模型（LLM）已借助文本知识图谱进行关系推理，但将图结构线性化为文本往往导致词元效率低下并丢失高阶关系线索。受大型多模态模型在显式捕捉高阶关系结构方面进展的启发，本文提出一种全新的可视化知识表征范式，即将知识图谱转换为图形化可视化，使大型多模态模型能够直接感知并推理。为系统评估这一能力，我们提出 VKG-QA，一个面向视觉知识图谱问答的基准，涵盖三大类别与十四个子任务。VKG-QA 通过半自动流程构建，确保数据高质量、语义对齐且视觉清晰。我们在 VKG-QA 上评测了 19 个具有代表性的大型多模态模型，并开展了大量定量与定性分析。结果表明，当前模型在可视化关系理解方面存在困难，针对图结构的理解仍具挑战性，且闭源模型显著优于开源模型。因此，VKG-QA 揭示了当前大型多模态模型的关键局限，并为推进具备图感知能力的视觉推理提供了一个可扩展的平台。
英文原文摘要 1365 字符
Understanding and reasoning over structured knowledge is a fundamental capability for intelligent systems. While Large Language Models (LLMs) have leveraged textual knowledge graphs for relational reasoning, linearizing graph structures into text often leads to token inefficiency and loss of higher-order relational cues. Inspired by the advances of Large Multimodal Model to capture higher-order relational structures explicitly novel paradigm of visualized knowledge representation, where knowledge graphs are transformed into graphical visualizations that LMMs can directly perceive and reason over. To systematically evaluate this capability, we introduce VKG-QA, a benchmark for Visual Knowledge Graph-based Question Answering, covering three major categories and fourteen subtasks. VKG-QA is constructed via a semi-automatic pipeline ensuring high-quality, semantically aligned, and visually clear data. We evaluate 19 representative LMMs on VKG-QA and perform extensive quantitative and qualitative analyses. Results reveal that current models struggle with visualized relational understanding, graph-specific comprehension remains challenging, and closed-source models significantly outperform open-source counterparts. VKG-QA thus highlights critical limitations in current LMMs and provides a scalable platform for advancing graph-aware visual reasoning.