词汇缩放定律：调整开放词表预测器以提升其开放性
Vocabulary Scaling Law: Tuning Open-vocabulary Predictors for Their Openness
提出词汇缩放定律，指导开放词表预测器的微调应覆盖整个词汇表、调整类名嵌入并强制提示词嵌入正交；SVFT 通过子模最大化选择信息量大的类名子集进行高效微调，可直接用于提升 CLIP 在开放世界中的稳定性和可扩展性。
原文依据 We formalize a "vocabulary scaling law" showing that these openness measures can be lower-bounded by performance on the full class-name universe
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于 CLIP 的开放词表学习在多样概念上展现出卓越的泛化能力，然而在现实的流式开放世界评估中，面对干扰类别的稳定性以及对新类别的可扩展性时却表现不佳。当前的微调方法往往无法通过这些测试，因为它们主要针对闭集条件设计，导致当目标词汇表逐步扩展时出现性能差距。我们形式化了一个“词汇缩放定律”，表明这些开放性度量可以由在整个类名全集上的性能下界所界定，这意味着鲁棒的微调应当：(i) 考虑整个词汇表，(ii) 调整类名嵌入而非上下文，以及 (iii) 强制提示词嵌入之间的正交性，包括训练类和开放集类名。在我们的分析指导下，我们提出了子模词汇微调（SVFT），这是一个双层优化框架，通过约束子模最大化贪婪地选择一个小型、信息量大的类名子集，来近似调整所有类名嵌入这一难以处理的目标，从而能够采用高效的贪婪算法进行接近最优的类名子集选择，以微调 CLIP，而不是使用所有开放类别。在大量实验中，SVFT 持续提升了稳定性和可扩展性，增强了基于 CLIP 的视觉语言模型的开放性和实际鲁棒性。
英文原文摘要 1441 字符
Open-vocabulary learning on CLIP provides remarkable generalization on diverse concepts, however, falters under the realistic streaming open-world evaluations for Stability against distractor classes and Extensibility to novel classes. Current fine-tuning methods often fail these tests since they are mainly designed for closed-set conditions, leading to the performance gaps while the target vocabulary progressively scales. We formalize a "vocabulary scaling law" showing that these openness measures can be lower-bounded by performance on the full class-name universe, implying that robust fine-tuning should: (i) account for the entire vocabulary, (ii) tune class-name embeddings rather than context, and (iii) enforce orthogonality between prompt embeddings including training and open-set class names. Guided by our analysis, we propose Submodular-Vocabulary Fine-tuning (SVFT), a bi-level optimization framework that approximates the intractable objective of tuning all class name embedding by greedily selecting a small, informative subset of class names via constrained submodular maximization, thus, allows the employment of efficient greedy algorithm for the near-optimal class-name subset selection to fine-tune CLIP instead of using all open classes. Across extensive experiments, SVFT consistently improves both stability and extensibility, advancing the openness and practical robustness of CLIP-based vision-language models.