TaskForce：面向多任务优化的协作式多智能体强化学习
TaskForce: Cooperative Multi-agent Reinforcement Learning for Multi-task Optimization
TaskForce 把多任务损失加权建模为协作式多智能体强化学习问题，智能体以梯度 Gram 矩阵和损失值为观测输出各任务的平衡权重；做多任务视觉模型（如 NYU-v2 深度+法向、Cityscapes 检测+分割）时可直接复用该梯度级冲突消解与动态加权机制。
原文依据 agents observe a summary of the training dynamics that consists of the gradient Gram matrix---capturing both gradient magnitudes and pairwise alignments---and task loss values
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多任务学习（MTL）涉及多个任务特定损失的同步优化，常导致梯度冲突与尺度失衡，进而产生负迁移。现有多种多任务优化方法虽试图缓解这些挑战，但要么缺乏逃离较差局部极小值所需的随机性，要么未能在梯度层面显式处理冲突。本文提出 TaskForce，一种融入协作式多智能体强化学习（MARL）的新型多任务优化框架，其中各智能体基于各自的任务梯度与损失学习寻找有效的联合优化策略。为使优化过程紧凑而信息充分，智能体观测由梯度 Gram 矩阵（同时刻画梯度幅值与两两对齐关系）和任务损失值构成的训练动态摘要。随后每个智能体预测平衡参数，以确定其对最终梯度更新贡献的权重。关键在于，我们设计了一种混合奖励函数，同时纳入基于梯度的信号与损失改善动态，使智能体能够兼顾直接的梯度信息及其对损失下降的实际影响，从而有效化解梯度冲突并避免较差的收敛。TaskForce 在 NYU-v2、Cityscapes 和 QM9 上相较最优 MTL 基线取得一致提升，展示了协作式 MARL 在复杂多任务场景中的潜力。
英文原文摘要 1528 字符
Multi-task learning (MTL) involves the simultaneous optimization of multiple task-specific losses, often leading to gradient conflicts and scale imbalances that result in negative transfer. While existing multi-task optimization methods attempt to mitigate these challenges, they either lack the stochasticity needed to escape poor local minima or fail to explicitly address conflicts at the gradient level. In this work, we propose TaskForce, a novel multi-task optimization framework incorporating cooperative multi-agent reinforcement learning (MARL), where agents learn to find an effective joint optimization strategy based on their respective task gradients and losses. To keep the optimization process compact yet informative, agents observe a summary of the training dynamics that consists of the gradient Gram matrix---capturing both gradient magnitudes and pairwise alignments---and task loss values. Each agent then predicts the balancing parameters that determine the weight of their contribution to the final gradient update. Crucially, we design a hybrid reward function that incorporates both gradient-based signals and loss improvement dynamics, enabling agents to effectively resolve gradient conflicts and avoid poor convergence by considering both direct gradient information and the resulting impact on loss reduction. TaskForce achieves consistent improvements over state-of-the-art MTL baselines on NYU-v2, Cityscapes, and QM9, demonstrating the promise of cooperative MARL in complex multi-task scenarios.