可解释性与可视化 · CVPR2026 ·2026.06.03

圆头还是流线型头?连接概念瓶颈模型与属性描述的物体部件

Rounded or Streamlined Head? Bridging Concept Bottleneck Models and Attribute-Described Object Parts

Yang Liu、Jiajin Zhang、Yaojun Hu、Bingguang Hao、Xin Cao、Yingda Xia 等 9 人

与你有关的部分

把概念重新定义为「部件-属性对」并用类别无关的目标聚类抑制背景伪证据,这套做法可直接迁移到任何需要 VLM 给出细粒度部件掩码的可解释识别或属性定位流程中,缓解 VLM 部件定位噪声。

原文依据 we redefine concepts as part-attribute pairs to enhance VLM robustness at the semantic level
可迁移到 可解释图像分类细粒度部件定位概念定位

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

一个忠实的决策过程要求模型在空间上(概念出现在图像的何处)和因果上(概念如何影响预测)对类人可理解的概念进行视觉定位。视觉语言模型(VLM)的最新进展使概念级对齐成为可能,并启发了概念瓶颈模型(CBM):这类模型通过将图像表征映射到人类可理解的概念来解释预测,使用户能够通过显式的语义推理追溯决策过程。然而,现有 CBM 存在两个关键的不一致。其一是语义不一致:VLM 往往无法定位细粒度的部件-属性概念,产生带噪声或不完整的掩码。其二是目标不一致:诸如「头部:流线型正面轮廓」这类与目标无关的概念可能描述多个类别(例如鱼或人);若不施加目标身份约束,非目标区域会引入虚假证据,从而污染瓶颈表征。为应对这些挑战,我们提出一种新的目标感知概念瓶颈模型(OA-CBM),同时施加语义层面与目标层面的一致性。具体而言,(1) 我们将概念重新定义为部件-属性对,以在语义层面增强 VLM 的鲁棒性;(2) 引入类别无关的目标聚类以抑制无关的视觉证据。我们进一步为两个视觉定位数据集标注了部件-属性描述,并开展了大量实验。结果表明,OA-CBM 在保持有竞争力的预测性能的同时,能产生更忠实、更鲁棒的解释。

英文原文摘要 1566 字符

A faithful decision-making process requires models to ground human-understandable concepts both spatially (where they appear in the image) and causally (how they influence the prediction). Recent advances in Vision-Language Models (VLMs) enable concept-level alignment and have inspired Concept Bottleneck Models (CBMs), which explain predictions by mapping image representations to human-understandable concepts, allowing users to trace decisions through explicit semantic reasoning. However, existing CBMs suffer from two key inconsistencies. First, semantic inconsistency: VLMs often fail to localize fine-grained part-attribute concepts, producing noisy or incomplete masks. Second, object inconsistency: object-agnostic concepts such as "head: streamlined front profile" may describe multiple categories (e.g., fish or human); without enforcing object identity, non-targeted regions can introduce spurious evidence that corrupts the bottleneck representation. To address these challenges, we propose a new Object-Aware Concept Bottleneck Model (OA-CBM) that jointly enforces semantic- and object-level consistency. Specifically, (1) we redefine concepts as part-attribute pairs to enhance VLM robustness at the semantic level, and (2) introduce class-agnostic object clustering to suppress irrelevant visual evidence. We further annotate two grounding datasets with part-attribute descriptions and conduct extensive experiments. Results demonstrate that OA-CBM produces more faithful and robust explanations while maintaining competitive predictive performance.

标签

会议 CVPR2026
页码 9880-9890
Bibkey
来源 CVF Open Access