双曲视觉语言模型中基于不确定性引导与部分到整体语义代表性的组合对齐
Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models
把双曲空间中的不确定性作为「部分对整体语义代表性」的度量，并用不确定性加权对比损失与熵正则的蕴含损失来校准，可直接迁移到需要建模层级/组合结构的图文对齐与检索任务。
原文依据 UNCHA models part-to-whole semantic representativeness with hyperbolic uncertainty
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管视觉语言模型（VLM）已取得显著性能，其欧氏嵌入在捕捉部分到整体、父子等层级关系方面仍然受限，并且在多目标组合场景中常常面临挑战。双曲 VLM 通过更好地保持层级结构、并借助蕴含关系建模部分-整体关系（即整体场景与其部分图像）来缓解该问题。然而，现有方法并未建模每个部分对整体具有不同水平的语义代表性。我们提出不确定性引导的组合双曲对齐（UNCHA）以增强双曲 VLM。UNCHA 用双曲不确定性建模部分到整体的语义代表性：对整体场景而言，为更具代表性的部分赋予更低的不确定性，为代表性较弱的部分赋予更高的不确定性。该代表性随后通过不确定性引导的权重被纳入对比目标。最后，该不确定性进一步用带有熵正则项的蕴含损失进行校准。借助所提出的损失，UNCHA 学习到具有更准确部分-整体排序的双曲嵌入，捕捉图像中潜在的组合结构，并提升其对复杂多目标场景的理解。UNCHA 在零样本分类、检索和多标签分类基准上取得最优性能。代码与模型见：https://github.com/jeeit17/UNCHA.git。
英文原文摘要 1537 字符
While Vision-Language Models (VLMs) have achieved remarkable performance, their Euclidean embeddings remain limited in capturing hierarchical relationships such as part-to-whole or parent-child structures, and often face challenges in multi-object compositional scenarios. Hyperbolic VLMs mitigate this issue by better preserving hierarchical structures and modeling part-whole relations (i.e., whole scene and its part images) through entailment. However, existing approaches do not model that each part has a different level of semantic representativeness to the whole. We propose UNcertainty-guided Compositional Hyperbolic Alignment (UNCHA) for enhancing hyperbolic VLMs. UNCHA models part-to-whole semantic representativeness with hyperbolic uncertainty, by assigning lower uncertainty to more representative parts and higher uncertainty to less representative ones for the whole scene. This representativeness is then incorporated into the contrastive objective with uncertainty-guided weights. Finally, the uncertainty is further calibrated with an entailment loss regularized with entropy-based term. With the proposed losses, UNCHA learns hyperbolic embeddings with more accurate part-whole ordering, capturing the underlying compositional structure in an image and improving its understanding of complex multi-object scenes. UNCHA achieves state-of-the-art performance on zero-shot classification, retrieval, and multi-label classification benchmarks. Our code and models are available at: https://github.com/jeeit17/UNCHA.git.