类 CLIP 模型作为基础密度比估计器
CLIP-like Model as a Foundational Density Ratio Estimator
把 CLIP/SigLIP 的对比学习目标重新解释为预训练密度比估计器，可直接用于多模态数据筛选（KL 引导的 LAION2B 过滤替代）和少样本提示词下的重要性加权，为视觉-语言数据处理与提示词工程提供可复用工具。
原文依据 we reinterpret CLIP-style models as pretrained and general-purpose density ratio estimators
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论密度比估计是统计机器学习的核心概念，因为它为何重要性加权、散度估计和无似然推断等任务提供了统一机制，但其在视觉与语言模型中的潜力尚未被充分探索。CLIP 和 SigLIP 等现代视觉-语言编码器以对比目标训练，这些目标隐式地优化联合图像-文本分布与边缘分布之间的对数密度比，从而隐式地学习与对数密度比成比例的相似度分数。然而，先前工作主要关注其嵌入用途，对比学习所诱导的密度比结构尚未在多模态应用中被系统考察或利用。为填补这一空白，我们将 CLIP 风格模型重新解释为预训练的通用密度比估计器，并表明这一视角能带来新的算法能力。我们统一解释了对比目标如何估计密度比，并提出两个实际应用：重要性权重学习与 KL 散度估计。我们的重要性权重学习方法仅需一个额外提示词，即可将 F1 分数最多提升 7 个点。我们进一步表明，基于 CLIP 的密度比支持估计 KL 散度，以量化以图像或文本为条件如何改变另一模态的分布。通过定性示例和对描述文本的 N-gram 分析，我们发现这些散度捕捉了多模态数据中的语义多样性和模态结构。利用这一性质，我们提出一种简单的 KL 引导数据筛选方法，其性能与 LAION2B 过滤相当。代码见 https://github.com/fumiyauchiyama/CLIP_Density_Ratio 。
英文原文摘要 1817 字符
Density ratio estimation is a core concept in statistical machine learning because it provides a unified mechanism for tasks such as importance weighting, divergence estimation, and likelihood-free inference, but its potential in vision and language models has not been fully explored. Modern vision-language encoders such as CLIP and SigLIP are trained with contrastive objectives that implicitly optimize log density ratios between joint and marginal image-text distributions, which implicitly learn similarity scores proportional to log density ratios. However, prior work has largely focused on their embedding utility, and the density-ratio structure induced by contrastive learning has not been systematically examined or exploited in multimodal applications. To address this gap, we reinterpret CLIP-style models as pretrained and general-purpose density ratio estimators and show that this perspective enables new algorithmic capabilities. We present a unified explanation of how contrastive objectives estimate density ratios and propose two practical applications: Importance Weight Learning and KL divergence estimation. Our Importance Weight Learning method requires only a single additional prompt and improves F1 scores by up to 7 points. We further show that CLIP-based density ratios support estimation of KL divergences that quantify how conditioning on an image or text alters the distribution of the other modality. Through qualitative examples and an N-gram analysis of captions, we find that these divergences capture semantic diversity and mode structure in multimodal data. Leveraging this property, we introduce a simple KL-guided data curation method that achieves performance competitive with LAION2B filtering. Our code is available at https://github.com/fumiyauchiyama/CLIP_Density_Ratio .