视觉语言模型 · CVPR2026 ·2026.06.03

VCU-Bridge:通过语义桥接的层次化视觉内涵理解

VCU-Bridge: Hierarchical Visual Connotation Understanding via Semantic Bridging

Ming Zhong、Yuanlei Wang、Liuzhou Zhang、Ruichuan An、Renrui Zhang、Hao Liang 等 9 人

与你有关的部分

提出层次化视觉内涵理解框架和HVCU-Bench基准,用于诊断MLLM从低层感知到高层推理的因果依赖;其MCTS引导的数据生成流程可复用于构建其他需要显式推理链的视觉语言指令微调数据。

原文依据 We present VCU-Bridge, a framework that operationalizes a human-like hierarchy of visual connotation understanding
可迁移到 视觉问答多模态推理视觉语言理解

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

多模态大语言模型(MLLMs)在基准测试中表现出色,但其处理范式与人类整合视觉信息的能力存在差异。人类能够自然地桥接细节与高层概念,而模型往往孤立地处理这些元素。主流的评估协议通常将低层感知与高层推理解耦,忽视了二者之间的语义与因果依赖,导致评估结果缺乏诊断性,并掩盖了性能瓶颈。我们提出VCU-Bridge框架,将类人的视觉内涵理解层次体系操作化:从基础感知出发,经由语义桥接,推进至抽象内涵的多级推理,并形成从具体线索到抽象结论的显式证据-推理轨迹。基于该框架,我们构建了HVCU-Bench,一个具有显式逐级诊断功能的层次化视觉内涵理解基准。综合实验表明,随着推理向更高层级推进,性能持续下降。我们进一步开发了由蒙特卡洛树搜索(MCTS)引导的指令微调数据生成流程,并证明增强低层能力可在更高层级带来可测量的增益。有趣的是,该方法不仅在HVCU-Bench上有所提升,在通用基准上也带来收益(平均+2.53%),尤其在MMStar上获得显著提升(+7.26%),这证明了层次化思维模式的重要性及其在增强MLLM能力方面的有效性。

英文原文摘要 1581 字符

While Multimodal Large Language Models (MLLMs) excel on benchmarks, their processing paradigm differs from the human ability to integrate visual information. Unlike humans who naturally bridge details and high-level concepts, models tend to treat these elements in isolation. Prevailing evaluation protocols often decouple low-level perception from high-level reasoning, overlooking their semantic and causal dependencies, which yields non-diagnostic results and obscures performance bottlenecks. We present VCU-Bridge, a framework that operationalizes a human-like hierarchy of visual connotation understanding: multi-level reasoning that advances from foundational perception through semantic bridging to abstract connotation, with an explicit evidence-to-inference trace from concrete cues to abstract conclusions. Building on this framework, we construct HVCU-Bench, a benchmark for hierarchical visual connotation understanding with explicit, level-wise diagnostics. Comprehensive experiments demonstrate a consistent decline in performance as reasoning progresses to higher levels. We further develop a data generation pipeline for instruction tuning guided by Monte Carlo Tree Search (MCTS) and show that strengthening low-level capabilities yields measurable gains at higher levels. Interestingly, it not only improves on HVCU-Bench but also brings benefits on general benchmarks (average +2.53%), especially with substantial gains on MMStar (+7.26%), demonstrating the significance of the hierarchical thinking pattern and its effectiveness in enhancing MLLM capabilities.

标签

会议 CVPR2026
页码 26187-26197
Bibkey
来源 CVF Open Access