概念感知批采样改进语言-图像预训练
Concept-Aware Batch Sampling Improves Language-Image Pretraining
CABS 把「批采样」当作可在线定制的数据筛选手段：按目标概念分布动态组批（多样性最大化或频次最大化），可直接嵌入 CLIP 类对比学习训练循环，用于缓解长尾概念偏置，且无需额外模型过滤器。
原文依据 a simple yet effective batch-sampling framework that flexibly constructs batches on-the-fly based on specific target distributions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型应当在什么样的数据上训练？为回答这一问题，许多数据筛选工作都聚焦于数据集的质量。然而，现有方法大多（i）是离线的，即依据一组预先确定的过滤准则产出一个静态数据集；（ii）是概念无关的，即使用基于模型的过滤器，从而引入额外的数据集偏置。在本工作中，我们超越此类离线、概念无关的方法，倡导更灵活、任务自适应的在线概念式筛选。我们的第一项贡献是 DataConcept，一个包含 1.28 亿网络爬取图文对、并标注了其概念构成细粒度细节的集合。基于 DataConcept，我们提出 CABS（概念感知批采样），一个简单而有效的批采样框架，可依据特定目标分布即时灵活地构造批次。我们提出两个变体：（i）多样性最大化，用于构造对可用概念覆盖最广的批次；（ii）CABS-FM（频次最大化），用于构造物体多重性最大的批次。通过在四个视觉骨干网络和 28 个基准上的大量评测，我们证明 CABS 显著有益于语言-图像预训练（LIP），并在长尾评测上得到高性能模型。总体而言，CABS 是专有在线筛选算法的一个强有力的开源替代方案，使从业者能够定义自定义的概念分布，以针对特定下游任务进行优化。DataConcept 与 CABS 的源代码都将公开。
英文原文摘要 1613 字符
What data should a vision-language model be trained on? To answer this question, many data curation efforts center on the quality of a dataset. However, most of these existing methods are (i) offline, i.e. they produce a static dataset from a set of predetermined filtering criteria, and (ii) concept-agnostic, i.e. they use model-based filters which induce additional dataset bias. In this work, we go beyond such offline, concept-agnostic methods and advocate for more flexible, task-adaptive online concept-based curation. Our first contribution is DataConcept, a collection of 128M web-crawled image-text pairs annotated with fine-grained details about their concept composition. Building on DataConcept, we introduce CABS (Concept-Aware Batch Sampling), a simple yet effective batch-sampling framework that flexibly constructs batches on-the-fly based on specific target distributions. We propose two variants: (i) Diversity Maximization, to curate batches with the broadest coverage of available concepts, and (ii) CABS-FM (Frequency Maximization), to curate batches with maximal object multiplicity. Through extensive evaluations with four visual backbones and a suite of 28 benchmarks, we demonstrate that CABS significantly benefits Language-Image Pretraining (LIP) and yields highly performant models on long-tailed evaluations. Overall, CABS represents a strong open-source alternative to proprietary online curation algorithms, enabling practitioners to define custom concept distributions that optimize for specific downstream tasks. Both DataConcept and the source code for CABS will be made public.