DynamicGTR：利用图拓扑表示偏好提升视觉语言模型在图问答上的能力
DynamicGTR: Leveraging Graph Topology Representation Preferences to Boost VLM Capabilities on Graph QAs
对做 VLM 的人来说，这篇给出了一个可复用的思路：把结构化输入（图）的多种表示形式（图像式、文本式）当作可选择的「模态」，在推理时按查询动态挑选，从而在准确率与输出长度之间做可控权衡；这种「表示选择器 + 零样本迁移」的框架可以迁移到视觉侧的结构化输入（如场景图、流程图、图表）的 VLM 问答上。
原文依据 dynamically selects the optimal GTR for each query during inference
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）已成为跨多个领域进行零样本问答（QA）的通用解决方案。然而，让 VLM 有效理解结构化图并执行准确、高效的问答仍然具有挑战性。现有方法通常依赖单一类型的图拓扑表示（GTR），例如固定风格的视觉图像或统一的文本描述。这种「一刀切」策略往往忽略了模型特定与任务特定的偏好，导致对图相关查询的回答不准确或过于冗长。为解决这一问题，我们提出 DynamicGTR 框架，它在推理过程中为每个查询动态选择最优的 GTR，从而以可定制的准确率与简洁性权衡增强 VLM 的零样本图问答能力。大量实验表明，DynamicGTR 不仅提升了基于 VLM 的图算法问答性能，还能在无需任何额外训练的情况下，将从合成图算法任务中训练得到的经验迁移到链路预测和节点分类等真实世界应用中。此外，DynamicGTR 在任务、领域和模型之间展现出很强的可迁移性，表明其有潜力成为面向广泛图场景的灵活解决方案。
英文原文摘要 1303 字符
Vision-Language Models (VLMs) have emerged as versatile solutions for zero-shot question answering (QA) across various domains. However, enabling VLMs to effectively comprehend structured graphs and perform accurate, efficient QA remains challenging. Existing approaches typically rely on a single type of graph topology representation (GTR) of graphs, such as fixed-style visual images or unified text descriptions. This "one-size-fits-all" strategy often neglects model-specific and task-specific preferences, resulting in inaccurate or overly lengthy responses to graph-related queries. To address this, we propose the DynamicGTR framework, which dynamically selects the optimal GTR for each query during inference, thereby enhancing the zero-shot graph QA capabilities of VLMs with a customizable accuracy and brevity trade-off. Extensive experiments show that DynamicGTR not only improves VLM-based graph algorithm QA performance but also successfully transfers the experience trained from synthetic graph algorithm tasks to real-world applications like link prediction and node classification, without any additional training. Additionally, DynamicGTR demonstrates strong transferability across tasks, domains, and models, suggesting its potential as a flexible solution for broad graph scenarios.