理解并强制任务算术中的权重解耦
Understanding and Enforcing Weight Disentanglement in Task Arithmetic
对做视觉模型微调与多任务合并的人，OrthoReg 提供了一个可直接插入微调过程的权重更新正交正则项，用来缓解任务向量相加时的相互干扰，可用于多任务模型合并、个性化编辑与持续微调场景。
原文依据 we propose OrthoReg, a simple and effective regularization method that actively enforces an internal orthogonal structure on weight updates
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论任务算术提供了一种高效、免训练的方式来编辑预训练模型，但其成功缺乏根本性的理论解释。现有的“权重解耦”概念描述了无干扰任务组合的理想结果，却未揭示其内在原因。关键在于，预训练模型（θ0）或任务向量（τt）的哪些内在性质使这种解耦成为可能，目前仍缺乏深入探索。本文引入任务-特征特化（Task-Feature Specialization, TFS），即模型将不同的内部特征分配给不同任务的能力，作为其基本原理。我们首先证明 TFS 是权重解耦的充分条件。更重要的是，我们发现 TFS 还带来一个可观测的几何结果：权重向量正交性。这使 TFS 成为期望的功能结果（解耦）与可度量的几何性质（正交性）的共同原因。这一关系为我们的方法提供了关键洞见：由于抽象的 TFS 性质难以直接施加，我们可以转而通过塑造其具体的几何结果——正交性——来促进权重解耦。因此，我们提出 OrthoReg，一种简单而有效的正则化方法，在微调过程中主动对构成 τt 的权重更新（ΔW）施加内部正交结构。我们从理论上证明 OrthoReg 能促进解耦。大量实验表明，OrthoReg 能持续且显著地提升多种任务算术方法的性能。
英文原文摘要 1622 字符
Task arithmetic provides an efficient, training-free way to edit pre-trained models, yet lacks a fundamental theoretical explanation for its success. The existing concept of "weight disentanglement" describes the ideal outcome of non-interfering task composition but does not reveal its underlying cause. Crucially, what intrinsic properties of the pre-trained model (\theta_0) or the task vectors (\tau_t) enable this disentanglement remains underexplored. In this paper, we introduce Task-Feature Specialization (TFS), a model's ability to allocate distinct internal features to different tasks, as the fundamental principle. We first prove that TFS is a sufficient condition for weight disentanglement. More importantly, we find that TFS also gives rise to an observable geometric consequence: weight vector orthogonality. This positions TFS as the common cause for both the desired functional outcome (disentanglement) and a measurable geometric property (orthogonality). This relationship provides the key insight for our method: since the abstract TFS property is intractable to enforce directly, we can instead promote weight disentanglement by shaping its concrete geometric consequence, orthogonality. Therefore, we propose OrthoReg, a simple and effective regularization method that actively enforces an internal orthogonal structure on weight updates (\Delta W) that constitute \tau_t during fine-tuning. And we theoretically prove that OrthoReg promotes disentanglement. Extensive experiments demonstrate that OrthoReg consistently and significantly enhances the performance of various task arithmetic methods.