b-CLIP：面向多粒度视觉-语言对齐的文本条件对比学习
b-CLIP: Text-Conditioned Contrastive Learning for Multi-Granular Vision-Language Alignment
用交叉注意力把图像 patch 按不同文本粒度（整句、句子、短语）动态池化成上下文视觉嵌入，并给出可调的多粒度对比损失，可直接用于需要密集图文对应的检索、grounding 与区域-短语匹配任务。
原文依据 b-CLIP uses cross-attention to dynamically pool image patches into contextualized visual embeddings
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论CLIP 通过对齐全局视觉与文本表示，在零样本图像-文本检索上表现强劲，但即便在长而详细的描述文本上微调，它在细粒度任务上仍然落后。本文提出 b-CLIP，一个多粒度文本条件对比学习框架，旨在实现多种文本粒度——从完整描述到句子和短语——与其对应视觉区域之间的层次化对齐。对于每一级粒度，b-CLIP 使用交叉注意力将图像块动态池化为上下文相关的视觉嵌入。为处理该层次结构固有的语义重叠，我们引入 b-Contextualized Contrastive Alignment Loss（b-CAL），在交叉熵与二元交叉熵两种形式下，将查询特定匹配与图像内上下文化之间的权衡参数化。通过大量实验，我们证明 b-CLIP 显著提升了密集对齐，在 Urban1K 上取得 R@1 的 91.8% T2I 与 92.3% I2T，在 FG-OVD（Hard）上取得 30.9%，在未使用难负样本训练的方法中达到最优。b-CLIP 为密集视觉-语言对应建立了一个稳健、自适应的基线。代码与模型已发布于 https://github.com/fzohra/B-CLIP。
英文原文摘要 1330 字符
CLIP achieves strong zero-shot image-text retrieval by aligning global vision and text representations, yet it falls behind on fine-grained tasks even when fine-tuned on long, detailed captions. In this work, we propose b-CLIP, a multi-granular text-conditioned contrastive learning framework designed to achieve hierarchical alignment between multiple textual granularities--from full captions to sentences and phrases--and their corresponding visual regions. For each level of granularity, b-CLIP uses cross-attention to dynamically pool image patches into contextualized visual embeddings. To address the semantic overlap inherent in this hierarchy, we introduce the b-Contextualized Contrastive Alignment Loss (b-CAL), which parameterizes the trade-off between query-specific matching and intra-image contextualization under both Cross-Entropy and Binary Cross-Entropy formulations. Through extensive experiments, we demonstrate that b-CLIP significantly improves dense alignment, achieving 91.8% T2I 92.3% I2T at R@1 on Urban1K and 30.9% on FG-OVD (Hard), setting state-of-the-art among methods trained without hard negatives. b-CLIP establishes a robust, adaptive baseline for dense vision-language correspondence. The code and models are released at \href https://github.com/fzohra/B-CLIP https://github.com/fzohra/B-CLIP .