无标签跨任务 LoRA 合并与零空间压缩
Label-Free Cross-Task LoRA Merging with Null-Space Compression
提出一种无需标签、基于适配器几何的 LoRA 合并方法，可直接用于视觉任务中多个微调模型的高效融合，避免联合训练开销。
原文依据 NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论模型合并将独立微调的检查点组合起来，无需联合多任务训练。在基础模型时代，使用低秩适配（LoRA）进行微调十分普遍，使得 LoRA 合并成为一个有前景的目标。现有方法可以在所有目标任务均为分类的同质设置中工作，但当任务跨越分类和回归时往往失败。使用基于熵的替代指标的方法不适用于回归，并且由于长词元序列而对大型语言模型成本高昂。我们引入零空间压缩（NSC）合并，这是一种无标签、输出无关的方法，从适配器几何中设置合并权重。我们的关键观察是，在 LoRA 微调过程中，下投影因子（A）在（ΔW = BA）中压缩其零空间，并且这种压缩与性能相关。NSC 将此作为合并的优化信号，可以泛化到分类、回归和序列生成。NSC 在二十个异构视觉任务上实现了最优性能，并取得了均衡的增益，而先前方法在任务子集上过拟合。它还在六个 NLI 基准以及 VQA 和图像描述的视觉-语言评估上优于基线，展示了可扩展性和有效性。我们的代码可在 https://github.com/wonyoung01/nsc_merging 获取。
英文原文摘要 1343 字符
Model merging combines independently fine-tuned checkpoints without joint multi-task training. In the era of foundation-model, fine-tuning with Low-Rank Adaptation (LoRA) is prevalent, making LoRA merging a promising target. Existing approaches can work in homogeneous settings where all target tasks are classification but often fail when tasks span classification and regression. Approaches using entropy-based surrogates do not apply to regression and are costly for large language models due to long token sequences. We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry. Our key observation is that during LoRA finetuning the down-projection factor (A) in (\Delta W = BA) compresses its null space, and the compression correlates with performance. NSC uses this as an optimization signal for merging that can generalize across classification, regression, and sequence generation. NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks. It also outperforms baselines on six NLI benchmarks and on vision-language evaluations for VQA and image captioning, demonstrating scalability and effectiveness. Our code is available at https://github.com/wonyoung01/nsc_merging.