黄金子空间:持续测试时自适应中效率与泛化的交汇点
The Golden Subspace: Where Efficiency Meets Generalization in Continual Test-Time Adaptation
把预训练分类器的行空间当作「黄金子空间」,用样本级平均梯度外积(AGOP)在线估计它,再用轻量适配器把特征投影进去——这套做法可以迁移到任何需要在无源数据、分布偏移下在线更新特征的视觉模型(检测、分割、BEV 感知),且只学一个紧凑缩放向量,几乎不增加推理开销。
原文依据 we introduce the sample-wise Average Gradient Outer Product (AGOP) as an efficient proxy for estimating the classifier weights without retraining
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论持续测试时自适应(CTTA)旨在使模型能够在分布偏移下、在不访问源数据的情况下,对无标签数据流进行在线自适应。现有 CTTA 方法面临效率与泛化之间的权衡:更新更多参数能提升自适应效果,却严重降低在线推理效率。理想的解决方案是以最小的特征更新获得相当的自适应效果;我们将这一最小子空间称为黄金子空间。我们证明了它在单步自适应设定下的存在性,并表明它与预训练分类器的行空间重合。为实现在线维护该子空间,我们引入样本级平均梯度外积(AGOP),作为无需重新训练即可估计分类器权重的高效代理。基于这些洞见,我们提出引导式在线低秩方向自适应(GOLD):它使用轻量适配器将特征投影到黄金子空间上,并学习一个紧凑的缩放向量,同时该子空间通过 AGOP 被动态更新。在包括自动驾驶场景在内的分类与分割基准上的大量实验表明,GOLD 在效率、稳定性与整体性能上均取得优越表现。代码已开源至 https://github.com/AIGNLAI/GOLD。
英文原文摘要 1296 字符
Continual Test-Time Adaptation (CTTA) aims to enable models to adapt online to unlabeled data streams under distribution shift without accessing source data. Existing CTTA methods face an efficiency-generalization trade-off: updating more parameters improves adaptation but severely reduces online inference efficiency. An ideal solution is to achieve comparable adaptation with minimal feature updates; we call this minimal subspace the golden subspace. We prove its existence in a single-step adaptation setting and show that it coincides with the row space of the pretrained classifier. To enable online maintenance of this subspace, we introduce the sample-wise Average Gradient Outer Product (AGOP) as an efficient proxy for estimating the classifier weights without retraining. Building on these insights, we propose Guided Online Low-rank Directional adaptation (GOLD), which uses a lightweight adapter to project features onto the golden subspace and learns a compact scaling vector while the subspace is dynamically updated via AGOP. Extensive experiments on classification and segmentation benchmarks, including autonomous-driving scenarios, demonstrate that GOLD attains superior efficiency, stability, and overall performance. Our code is available at https://github.com/AIGNLAI/GOLD.