CHIPS：基于曲率感知混合影响的数据选择实现高效 CLIP 适配
CHIPS: Efficient CLIP Adaptation via Curvature-aware Hybrid Influence-based Data Selection
把影响函数与曲率感知的对齐分数用在 CLIP 领域适配的数据筛选上，视觉研究者可直接复用这套打分流程，在医疗等垂直域只用 10%~30% 数据达到全量持续预训练的效果，也提供了投影子空间内估计 InfoNCE 曲率的可扩展做法。
原文依据 CHIPS attains state-of-the-art performance among selection baselines on 17 medical benchmarks, matches full-dataset CPT with 30% of the data
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论将 CLIP 适配到垂直领域通常采用新的微调策略，或在大规模领域特定数据集上进行持续预训练（CPT）。然而，数据本身在这一过程中仍是一个未被充分探索的因素。我们从以数据为中心的视角重新审视该任务：有效的数据选择能否在 CPT 中替代大规模数据集？我们提出 CHIPS（投影子空间中的曲率感知混合影响），为每个图文对赋予一个效用分数，该分数融合了与三个目标相对应的三个互补因素：忠实性，通过在 CLIP 端点子空间中计算的曲率感知牛顿式对齐来实现；可扩展性，通过带有 Johnson-Lindenstrauss（JL）草图化的 InfoNCE 感知曲率估计器来实现；保留性，通过选择感知的相关性权重并结合可学习性，以在目标域适配与通用域保持之间取得平衡。我们从理论上论证了该设计：证明了该代理指标与全参数对齐之间相关性的下界保证，并刻画了曲率混合与 JL 草图化所引入的偏差—方差权衡。我们在多种设置下对 CHIPS 进行了实证评估：1）在 17 个医学基准上，CHIPS 在各类选择基线中取得最优性能，用 30% 的数据即可匹配全数据集 CPT，仅用 10% 的数据便超过半数据集 CPT；2）在 31 个通用域基准上，CHIPS 在所有保留比例下均取得最小的性能下降。
英文原文摘要 1476 字符
Adapting CLIP to vertical domains is typically approached by novel fine-tuning strategies or by continual pre-training (CPT) on large domain-specific datasets. Yet, data itself remains an underexplored factor in this process. We revisit this task from a data-centric perspective: Can effective data selection substitute for large-scale datasets in CPT? We introduce CHIPS (Curvature-aware Hybrid Influence in Projection Subspace), which assigns each image-text pair a utility score that integrates three complementary factors aligned with three goals: faithfulness via a curvature-aware and Newton-style alignment computed in CLIP's end-point subspace; scalability via an InfoNCE-aware curvature estimator with Johnson-Lindenstrauss (JL) sketching; and retention via a selection-aware relevance weight combined with learnability to balance target adaptation against general-domain preservation. We justify this design theoretically by proving a lower-bound guarantee on the proxy's correlation with full-parameter alignment and by characterizing the bias-variance trade-offs introduced by curvature mixing and JL sketching. We evaluate CHIPS empirically across various settings: 1) CHIPS attains state-of-the-art performance among selection baselines on 17 medical benchmarks, matches full-dataset CPT with 30% of the data, and outperforms half-dataset CPT using only 10%; 2) on 31 general-domain benchmarks, CHIPS yields the least performance drop under all retention ratios.