解构理想噪声校正的失败:三重诊断
Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis
对做噪声标签/弱监督训练的视觉研究者:即使给定 oracle 噪声转移矩阵,基于 T 的校正方法仍会训练崩溃,说明不应再把精力放在提升 T 估计精度上,而应关注校正过程中的优化动力学与信息论上限,可据此重新设计损失或训练调度。
原文依据 Even under these ideal conditions, we observe that these methods still suffer from performance collapse during training.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于噪声转移矩阵(T)的统计一致方法为噪声标签学习(LNL)提供了有理论依据的解决方案,并保证收敛到最优的干净数据分类器。然而在实践中,这些方法往往不如样本选择等经验性方法,这一差距通常被归因于难以准确估计 T。人们普遍假设,只要给定完美的 T,噪声校正方法就能恢复其理论优势。本文对这一长期存在的假设进行了决定性的检验。我们在理想化条件下开展实验,为校正方法提供完美的、oracle 的噪声转移矩阵。即使在这些理想条件下,我们仍观察到这些方法在训练过程中出现性能崩溃。这有力地表明,失败从根本上并不是 T 估计问题,而是源于更深层的缺陷。为解释这一行为,我们提供了一个统一分析,将三个层面联系起来:宏观的收敛状态、微观的优化动力学,以及从噪声标签中可学习内容的信息论极限。这些结果共同对理想噪声校正为何失败给出了形式化解释,并为设计更可靠的噪声标签学习方法提供了具体指导。
英文原文摘要 1377 字符
Statistically consistent methods based on the noise transition matrix (T) offer a theoretically grounded solution to Learning with Noisy Labels (LNL), with guarantees of convergence to the optimal clean-data classifier. In practice, however, these methods are often outperformed by empirical approaches such as sample selection, and this gap is usually attributed to the difficulty of accurately estimating T. The common assumption is that, given a perfect T, noise-correction methods would recover their theoretical advantage. In this work, we put this longstanding hypothesis to a decisive test. We conduct experiments under idealized conditions, providing correction methods with a perfect, oracle transition matrix. Even under these ideal conditions, we observe that these methods still suffer from performance collapse during training. This compellingly demonstrates that the failure is not fundamentally a T-estimation problem, but stems from a more deeply rooted flaw. To explain this behaviour, we provide a unified analysis that links three levels: macroscopic convergence states, microscopic optimisation dynamics, and information-theoretic limits on what can be learned from noisy labels. Together, these results give a formal account of why ideal noise correction fails and offer concrete guidance for designing more reliable methods for learning with noisy labels.