通过损失景观几何重新审视长尾学习
Reframing Long-Tailed Learning via Loss Landscape Geometry
将长尾识别中的尾部性能退化建模为持续学习中的灾难性遗忘，并利用损失景观几何（平坦极小值）来缓解；其分组知识保持与分组锐度感知模块可复用于其他存在类别不平衡或任务序列的视觉任务。
原文依据 we posit that this dilemma stems from a phenomenon called "tail performance degradation" in continual learning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在长尾数据分布上平衡性能权衡仍然是一个长期存在的挑战。本文认为这一困境源于持续学习中的一种称为“尾部性能退化”的现象（模型倾向于在头部类别上严重过拟合，同时快速遗忘尾部类别），并从损失景观的角度提出解决方案。我们观察到不同类别在损失景观中具有不同的收敛点。此外，当模型陷入尖锐且非鲁棒的极小值，而非一个对所有类别都有益的共享平坦解时，这种分歧会加剧。鉴于此，我们提出了一个受持续学习启发的框架来防止“尾部性能退化”。为避免低效的逐类参数保留，提出了分组知识保持模块来记忆组特定的收敛参数，促进向共享解收敛。同时，我们的框架集成了一个分组锐度感知模块，通过显式处理损失景观的几何形状来寻求更平坦的极小值。值得注意的是，我们的框架既不需要外部训练样本也不需要预训练模型，从而促进了广泛适用性。在四个基准上的大量实验表明，相较于最优方法有显著的性能提升。代码可在 https://gkp-gsa.github.io/ 获取。
英文原文摘要 1407 字符
Balancing performance trade-off on long-tail data distributions remains a long-standing challenge. In this paper, we posit that this dilemma stems from a phenomenon called "tail performance degradation" in continual learning (the model tends to severely overfit on head classes while quickly forgetting tail classes) and pose a solution from a loss landscape perspective. We observe that different classes possess divergent convergence points in the loss landscape. Besides, this divergence is aggravated when the model settles into sharp and non-robust minima, rather than a shared and flat solution that is beneficial for all classes. In light of this, we propose a continual learning inspired framework to prevent "tail performance degradation". To avoid inefficient per-class parameter preservation, a Grouped Knowledge Preservation module is proposed to memorize group-specific convergence parameters, promoting convergence towards a shared solution. Concurrently, our framework integrates a Grouped Sharpness Aware module to seek flatter minima by explicitly addressing the geometry of the loss landscape. Notably, our framework requires neither external training samples nor pre-trained models, facilitating the broad applicability. Extensive experiments on four benchmarks demonstrate significant performance gains over state-of-the-art methods. The code is available at: https://gkp-gsa.github.io/.