一条更快的持续学习路径
A Faster Path to Continual Learning
对做视觉持续学习或任何需要反复微调的场景，C-Flat Turbo 把 C-Flat 每步三次额外梯度计算压到更少，可直接替换优化器降低训练开销；其「平坦性梯度随任务趋于稳定」的观察也可用于设计其他正则化训练策略的调度。
原文依据 C-Flat requires three additional gradient computations per iteration, imposing substantial overhead on the optimization process.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论持续学习（CL）旨在让神经网络在动态任务流上训练而不遗忘先前学到的知识。在基于优化的方法中，C-Flat 因其即插即用的特性以及能为新旧任务同时促成均匀低损失区域的能力，已成为一个有前景的解决方案。然而，C-Flat 每次迭代需要额外计算三次梯度，给优化过程带来可观的开销。在本工作中，我们提出 C-Flat Turbo，一个更快且更强的优化器，显著降低了训练成本。我们表明，与一阶平坦性相关的梯度包含相对于代理模型梯度的方向不变分量，使我们能够跳过扰动上升步骤中冗余的梯度计算。此外，我们观察到这些促进平坦性的梯度会随任务推进逐渐趋于稳定，这促使我们采用一种带自适应触发器的线性调度策略，为后续任务分配更大的 turbo 步长。实验表明，在广泛的持续学习方法上，C-Flat Turbo 比 C-Flat 快 1.0 倍到 1.25 倍，同时取得相当甚至更高的准确率。
英文原文摘要 1184 字符
Continual Learning (CL) aims to train neural networks on a dynamic stream of tasks without forgetting previously learned knowledge. Among optimization-based approaches, C-Flat has emerged as a promising solution due to its plug-and-play nature and its ability to encourage uniformly low-loss regions for both new and old tasks. However, C-Flat requires three additional gradient computations per iteration, imposing substantial overhead on the optimization process. In this work, we propose C-Flat Turbo, a faster yet stronger optimizer that significantly reduces the training cost. We show that the gradients associated with first-order flatness contain direction-invariant components relative to the proxy-model gradients, enabling us to skip redundant gradient computations in the perturbed ascent steps. Moreover, we observe that these flatness-promoting gradients progressively stabilize across tasks, which motivates a linear scheduling strategy with an adaptive trigger to allocate larger turbo steps for later tasks. Experiments show that C-Flat Turbo is 1.0x to 1.25x faster than C-Flat across a wide range of CL methods, while achieving comparable or even improved accuracy.