FAAR：通过自动秩选择的高效频率感知多任务微调
FAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection
提出频率感知的自动秩选择方法，可复用于视觉多任务微调，在减少参数量的同时提升密集预测任务性能。
原文依据 FAAR reduces the number of parameters by up to 9 times compared to traditional MTL fine-tuning whilst improving overall performance.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在大规模数据集上预训练的模型进行适配已被证明是快速达到下游任务强性能的有效方法。然而，最优模型规模的增长使得传统的全量微调变得不合适且困难，尤其是在多任务学习（MTL）中，成本随任务数量增加而增长。因此，近期研究探索了使用低秩适配的参数高效微调（PEFT），以显著减少可训练参数的数量。然而，这些现有方法使用单一固定的秩，这可能对于不同任务或MTL架构中的不同位置并非最优。此外，这些方法未能学习到捕捉任务间关系并有助于改进多样化任务预测的空间信息。本文提出频率感知与自动秩（FAAR）方法，用于高效的MTL微调。我们的方法引入性能驱动的秩收缩（PDRS），为每个适配器位置和每个任务分配最优秩。此外，通过分析图像频谱，FAAR提出任务-频谱金字塔解码器（TS-PD），将输入特定的上下文注入空间偏置学习，以更好地反映跨任务关系。在密集视觉任务基准上的实验表明，与其他PEFT方法在MTL中相比，我们的方法在准确性和效率方面均具有优越性。FAAR相比传统MTL微调最多减少9倍的参数量，同时提升整体性能。我们的代码已公开。
英文原文摘要 1531 字符
Adapting models pre-trained on large-scale datasets is a proven way to reach strong performance quickly for downstream tasks. However, the growth of state-of-the-art models makes traditional full fine-tuning unsuitable and difficult, especially for multi-task learning (MTL) where cost scales with the number of tasks. As a result, recent studies investigate parameter-efficient fine-tuning (PEFT) using low-rank adaptation to significantly reduce the number of trainable parameters. However, these existing methods use a single, fixed rank, which may not be optimal for different tasks or positions in the MTL architecture. Moreover, these methods fail to learn spatial information that captures inter-task relationships and helps to improve diverse task predictions. This paper introduces Frequency-Aware and Automatic Rank (FAAR) for efficient MTL fine-tuning. Our method introduces Performance-Driven Rank Shrinking (PDRS) to allocate the optimal rank per adapter location and per task. Moreover, by analyzing the image frequency spectrum, FAAR proposes a Task-Spectral Pyramidal Decoder (TS-PD) that injects input-specific context into spatial bias learning to better reflect cross-task relationships. Experiments performed on dense visual task benchmarks show the superiority of our method in terms of both accuracy and efficiency compared to other PEFT methods in MTL. FAAR reduces the number of parameters by up to 9 times compared to traditional MTL fine-tuning whilst improving overall performance. Our code is available.