面向测试时自适应分割的原型混合专家
Mixture of Prototypes for Test-time Adaptive Segmentation
把类别原型聚类成多个域专家并用门控网络按实例-专家相关性加权，配合最小化预测熵、最大化专家选择熵的门控优化，可直接迁移到任何需要测试时自适应的分割或密集预测模型上。
原文依据 we devise a min-max entropy optimization scheme for the gating network but keeping the rest frozen
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论测试时自适应分割（TTA-Seg）旨在以无监督方式，使已训练的分割模型适应分布偏移下的测试数据。现有方法通常利用类别原型来捕获并迁移源分布，但不可避免地忽略了源样本内部的多样性。本文提出一种基于混合专家（MoE）的测试时自适应新范式，其中领域专家被设计用于：1）更好地捕获源分布；2）在测试样本预测中动态调整其贡献。具体而言，在源域训练阶段，原型由源像素特征的类别均值得到。随后通过对这些原型进行聚类生成多个专家，从而为每个类别提供若干更具代表性的专家。在测试阶段，每个实例的预测以自适应方式从所有专家的知识中得出，即由门控网络根据实例与专家的相关性分配权重。为优化该系统，我们设计了一种最小最大熵优化方案，仅作用于门控网络而保持其余部分冻结，即最小化模型预测的熵，同时最大化专家选择中的熵。由此，模型被促使在有效利用领域专家的同时给出置信的预测，从而促进自适应。在测试时自适应（TTA）以及更具挑战性的持续 TTA 两种场景上的实验表明，本方法取得了新的最优性能。
英文原文摘要 1542 字符
Test-Time Adaptive Segmentation (TTA-Seg) aims to adapt a trained segmentation model to test data under distribution shift in an unsupervised manner. Existing approaches typically utilize class-wise prototypes to capture and transfer the source distribution, but inevitably neglect the diversity within source samples. In this paper, we propose a new test-time adaptation paradigm based on the mixture-of-experts (MoE), where domain experts are designed to 1) better capture the source distribution, and 2) dynamically adjust their contribution in test case prediction. Specifically, during source training, prototypes are derived as the class-wise average for source pixel features. We then generate multiple experts through clustering these prototypes, providing each class with several experts with enhanced representativeness. At test time, each instance prediction is drawn from all experts' knowledge in an adaptive manner, i.e., a gating network assigns weights according to instance-expert correlations. To optimize the system, we devise a min-max entropy optimization scheme for the gating network but keeping the rest frozen, minimizing the entropy of model prediction but maximizing the entropy in expert selection. Consequently, the model is urged to derive confident predictions with effective utilization of domain experts, hence promoting the adaptation.Experiments on two scenarios, Test-time Adaptation (TTA) and the more challenging continual TTA, demonstrate that our approach achieves the new state-of-the-art performance.