测试时的可控联邦提示词学习
Controllable Federated Prompt Learning at Test Time
对做 CLIP 类视觉语言模型落地的人来说，这篇给出了一个测试时无需标签、仅靠置信度与模型-数据对齐指标动态加权全局/个性化/冻结三种提示词的方案，可直接迁移到跨域部署时的提示词自适应。
原文依据 we introduce a novel confidence-guided Model-Data Alignment (MoDA) metric in COTE that quantifies alignment at both macro and micro levels
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论联邦提示词学习（FPL）近期因其能够在联邦学习框架内利用 CLIP 等大规模视觉语言模型而受到越来越多的关注。尽管已有研究通过个性化策略推进了 FPL，以提升各客户端的专属性能，但个性化模型在跨未见域部署时，常因分布偏移而出现严重性能退化。本文首次探索测试时联邦提示词学习（TTFPL），旨在以最小代价弥合跨域性能差距，且仅需无标签的目标域数据。我们提出 COTE，一个三提示词可控 TTFPL 框架，动态平衡三种互补的提示词：来自标准 FPL 的全局提示词、来自个性化 FPL 的局部提示词，以及冻结的 CLIP 提示词。具体而言，我们在 COTE 中引入一种新颖的置信度引导的模型-数据对齐（MoDA）指标，在宏观与微观两个层面量化对齐程度，刻画模型预测与数据分布之间的一致性。通过将 MoDA 与模型置信度相结合，COTE 在测试时自适应地调整各提示词的贡献，从而在无需标注数据的条件下实现对异构客户端与未见域的稳健泛化。在多个基准数据集上的大量实验表明，我们的方法持续提升目标域性能，为自适应 FPL 开辟了新方向。
英文原文摘要 1457 字符
Federated Prompt Learning (FPL) has recently attracted increasing attention for its ability to leverage large-scale vision-language models such as CLIP within federated learning frameworks. While existing studies have advanced FPL through personalization strategies to enhance client-specific performance, personalized models often suffer severe degradation when deployed across unseen domains due to distribution shifts.In this paper, we take the first step toward exploring Test-Time FPL (TTFPL), aiming to bridge the cross-domain performance gap with minimal effort, requiring only unlabeled target-domain data. We propose COTE, a tri-prompt controllable TTFPL framework that dynamically balances three complementary prompts: the global prompt from standard FPL, the local prompt from personalized FPL, and the frozen CLIP prompt.Specifically, we introduce a novel confidence-guided Model-Data Alignment (MoDA) metric in COTE that quantifies alignment at both macro and micro levels, capturing the consistency between model predictions and data distributions. By integrating MoDA with model confidence, COTE adaptively adjusts the contribution of each prompt at test time, enabling robust generalization across heterogeneous clients and unseen domains without requiring labeled data.Extensive experiments on multiple benchmark datasets demonstrate that our method consistently improves target-domain performance, setting a new direction for adaptive FPL.