面向基的低秩迁移用于少样本与测试时自适应
Basis-Oriented Low-rank Transfer for Few-Shot and Test-Time Adaptation
BOLT 提供了一种无需元训练、仅利用多个任务向量构造正交低秩基的迁移方法，视觉研究者可将其用于少样本或测试时快速适配新任务，尤其适合在已有多个微调模型（如不同数据集上的 ViT）时复用其知识。
原文依据 extracting an orthogonal, task-informed spectral basis and adapting within that subspace
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在数据和计算预算紧张的情况下，将大型预训练模型适配到未见任务仍然具有挑战性。元学习方法显式地学习良好的初始化，但需要在许多任务上进行额外的元训练阶段，产生高昂的训练成本，并且可能不稳定。同时，任务特定的预训练模型数量持续增长，然而如何以最小的额外训练将它们迁移到新任务的问题相对而言仍未得到充分探索。我们提出 BOLT（面向基的低秩迁移），一个通过提取正交的、任务感知的谱基并在该子空间内进行适配来复用现有微调模型的框架，而非合并权重。在离线阶段，BOLT 从多个任务向量中收集主导奇异方向，并逐层正交化以形成可复用的基。在在线阶段，我们冻结这些基，仅针对新任务逐层训练少量对角系数，产生一个秩受控的更新，可训练参数极少。这种设计提供了：(i) 针对未见任务的强免训练初始化，通过池化源任务系数并辅以轻量级重缩放步骤，同时利用共享的正交基；(ii) 一条参数高效微调（PEFT）路径，在我们的实验中，与常见的 PEFT 基线以及一个代表性的元学习初始化相比，取得了稳健的性能。我们的结果表明，将适配约束到任务感知的正交子空间为未见任务迁移提供了一种有效的替代方案。
英文原文摘要 1625 字符
Adapting large pre-trained models to unseen tasks under tight data and compute budgets remains challenging. Meta-learning approaches explicitly learn good initializations, but they require an additional meta-training phase over many tasks, incur high training cost, and can be unstable. At the same time, the number of task-specific pre-trained models continues to grow, yet the question of how to transfer them to new tasks with minimal additional training remains relatively underexplored. We propose BOLT (Basis-Oriented Low-rank Transfer), a framework that reuses existing fine-tuned models not by merging weights, but instead by extracting an orthogonal, task-informed spectral basis and adapting within that subspace.In the offline phase, BOLT collects dominant singular directions from multiple task vectors and orthogonalizes them per layer to form reusable bases. In the online phase, we freeze these bases and train only a small set of diagonal coefficients per layer for the new task, yielding a rank-controlled update with very few trainable parameters. This design provides (i) a strong, training-free initialization for unseen tasks, obtained by pooling source-task coefficients--along with a lightweight rescaling step--while leveraging the shared orthogonal bases, and (ii) a parameter-efficient fine-tuning (PEFT) path that, in our experiments, achieves robust performance compared to common PEFT baselines as well as a representative meta-learned initialization. Our results show that constraining adaptation to a task-informed orthogonal subspace provides an effective alternative for unseen-task transfer.