面向层次化视觉识别的分类学感知表征对齐：结合大型多模态模型
Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models
把生物基础模型经层次对比学习得到的表征，通过中间层特征对齐注入 LMM，为视觉研究者提供了一种无需重训即可让多模态模型获得分类学结构先验、并泛化到新类别的可复用对齐手段。
原文依据 aligning the intermediate representations of visual features with those of BFMs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论一个高性能的通用视觉理解模型应当能够将视觉输入映射到标签构成的分类学树，并识别训练集之外、几乎没有或完全没有公开图像的新类别。大型多模态模型（LMM）在已知类别的细粒度视觉识别（FGVR）上已取得显著进展。然而，它们在层次化视觉识别（HVR）上仍然受限，该任务旨在从粗到细地预测一致的标签路径，对新类别尤其如此。为应对这些挑战，我们提出分类学感知表征对齐（TARA），一种简单而有效的策略，用于将分类学知识注入 LMM。TARA 利用生物基础模型（BFM）的表征，这些表征通过层次对比学习编码了丰富的生物关系。通过将视觉特征的中间表征与 BFM 的表征对齐，LMM 被促使提取在分类学树中结构良好的判别性视觉线索。此外，我们将首个答案词元的表征与真实标签对齐，从而根据用户意图灵活弥合上下文视觉特征与不同粒度类别之间的差距。实验表明，TARA 持续提升了 LMM 的层次一致性与叶节点准确率，使其能够在复杂的生物分类学中可靠地识别已知与新颖类别。代码见 https://github.com/PKU-ICST-MIPL/TARA_CVPR2026。
英文原文摘要 1535 字符
A high-performing, general-purpose visual understanding model should map visual inputs to a taxonomic tree of labels, identify novel categories beyond the training set for which few or no publicly available images exist. Large Multimodal Models (LMMs) have achieved remarkable progress in fine-grained visual recognition (FGVR) for known categories. However, they remain limited in hierarchical visual recognition (HVR) that aims at predicting consistent label paths from coarse to fine categories, especially for novel categories. To tackle these challenges, we propose Taxonomy-Aware Representation Alignment (TARA), a simple yet effective strategy to inject taxonomic knowledge into LMMs. TARA leverages representations from biology foundation models (BFMs) that encode rich biological relationships through hierarchical contrastive learning. By aligning the intermediate representations of visual features with those of BFMs, LMMs are encouraged to extract discriminative visual cues well structured in the taxonomy tree. Additionally, we align the representations of the first answer token with the ground-truth label, flexibly bridging the gap between contextualized visual features and categories of varying granularity according to user intent. Experiments demonstrate that TARA consistently enhances LMMs' hierarchical consistency and leaf node accuracy, enabling reliable recognition of both known and novel categories within complex biological taxonomies. Code is available at https://github.com/PKU-ICST-MIPL/TARA_CVPR2026.