面向长尾视觉识别的可信校准协同学习
Trust-calibrated Collaborative Learning for Long-Tailed Visual Recognition
针对多专家互蒸馏中的错误传播与错误固化问题,提出知识质量门控与尾部类补偿机制,可迁移到任何采用多专家/集成蒸馏的视觉识别框架中,用于抑制专家间错误共识。
原文依据 indiscriminate knowledge transfer leads to bias propagation, where a single expert's error can spread and contaminate others
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论真实世界的视觉识别面临长尾分布这一根本性挑战。现有最优方法常采用多专家模型来处理不同频次的类别,但我们发现这些模型中使用的互知识蒸馏在增强协同性的同时,也带来了两个关键局限:不加区分地知识迁移会导致偏差传播,即单个专家的错误会扩散并污染其他专家;以及错误固化,即错误预测之间的相互强化会巩固错误的共识。为克服这些问题,我们提出可信校准协同学习(TCL)。该框架引入可信知识编排模块,通过知识质量门控阻断错误信息,并借助尾部类补偿机制缓解尾部类别的知识稀缺,从而实现可靠的蒸馏与精确的协同。此外,我们设计了共识误差校准模块,抑制共识高置信度负类,以纠正集体误判并将优化引向正确方向。在五个长尾基准上的大量实验表明,TCL 取得了最佳性能,将 CIFAR100-LT 上的 Top-1 准确率提升至 58.7%,较此前最优方法提升 2.4%。
英文原文摘要 1356 字符
Real-world visual recognition faces the fundamental challenge of long-tailed distributions. While state-of-the-art methods often employ multi-expert models to address different frequency categories, we find that the mutual knowledge distillation used in these models enhances collaboration at the cost of introducing two critical limitations: indiscriminate knowledge transfer leads to bias propagation, where a single expert's error can spread and contaminate others, and error consolidation, where mutual reinforcement of incorrect predictions solidifies erroneous consensus. To overcome these issues, we propose Trust-calibrated Collaborative Learning (TCL). Our framework introduces the trustworthy knowledge orchestration module, which enables reliable distillation and precise collaboration through a knowledge quality gate that blocks erroneous information and a tail-class compensation mechanism that alleviates knowledge scarcity for tail categories. Furthermore, we design a consensus error calibration module that suppresses consensus high-confidence negative classes to correct collective misjudgments and steer optimization in the right direction. Extensive experiments on five long-tailed benchmarks demonstrate that TCL achieves the best performance, raising Top-1 accuracy on CIFAR100-LT to 58.7%, a gain of 2.4% over previous SOTA methods.