面向视觉语言模型测试时自适应的动态 Logits 调整与探索
Dynamic Logits Adjustment and Exploration for Test-Time Adaptation in Vision Language Models
针对 VLM 测试时自适应中熵过滤只取高置信样本、继承类别偏置的问题，提出用测试预测统计动态校准 logits，并用一致性引导的探索缓存主动纳入决策边界附近样本；这套「logit 校准 + 边界探索缓存」可直接迁移到 CLIP 类模型的在线部署与分布偏移场景。
原文依据 DLA dynamically recalibrates model logits based on test prediction statistics, thereby mitigating class-wise prediction inconsistencies.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有的视觉语言模型（VLM）测试时自适应（TTA）方法主要致力于设计高效的适配参数（例如提示或残差原型），并普遍依赖通过熵过滤获得的高置信样本。然而，这一主流范式隐式地继承了 VLM 的类别级预测偏置，并导致对测试分布的覆盖不足，使自适应过程存在偏差且探索性不足。为克服这些局限，我们提出动态 Logits 调整与探索（DLAE）框架，将动态 Logits 调整（DLA）与一致性引导的探索缓存（CGEC）相结合。DLA 基于测试预测统计动态地重新校准模型 logits，从而缓解类别级预测不一致问题。与传统缓存机制不同，我们的 CGEC 主动识别决策边界附近、其预测标签对 logits 调整敏感的额外样本，从而在高置信样本之外进行探索。通过施加语义一致性与时序一致性约束，该缓存在保持所选样本可靠性的同时，能够对低置信区域进行谨慎而有效的探索，最终实现稳定可靠的自适应。在多个视觉语言基准上的大量实验表明，我们的方法持续超越最优的 TTA 方法，展现出更优的稳定性、适应性与泛化能力。
英文原文摘要 1502 字符
Existing Test-Time Adaptation (TTA) methods for Vision-Language Models (VLMs), focusing on designing efficient adaptation parameters (eg. prompts or residual prototypes), predominantly rely on high-confidence samples obtained via entropy-based filtering. However, this prevailing paradigm implicitly inherits the VLM's class-wise prediction biases and leads to insufficient coverage of the test distribution, rendering the adaptation process biased and insufficiently exploratory.To overcome these limitations, we propose Dynamic Logits Adjustment and Exploration (DLAE), a novel framework that integrates Dynamic Logit Adjustment (DLA) with a Consistency-Guided Exploratory Cache (CGEC). DLA dynamically recalibrates model logits based on test prediction statistics, thereby mitigating class-wise prediction inconsistencies. Different from traditional cache mechanisms, our CGEC actively identifies additional samples near decision boundaries whose predicted labels are sensitive to the logit adjustment, thereby exploring beyond only high-confidence samples. By enforcing semantic and temporal consistency, the cache preserves the reliability of selected samples while enabling cautious yet effective exploration of low-confidence regions, ultimately yielding stable and reliable adaptation.Extensive experiments across multiple vision-language benchmarks demonstrate that our approach consistently surpasses state-of-the-art TTA methods, showing superior stability, adaptability, and generalization.