HeeSun Bae, Seungjae Shin, Byeonghu Na, and Il-Chul Moon, Dirichlet-based Per-Sample Weighting by Transition Matrix for Noisy Label Learning, International Conference on Learning Representations (ICLR 2024), Vienna, Austria, May 7-11, 2024

Abstract

For learning with noisy labels, the transition matrix, which explicitly models the relation between noisy label distribution and clean label distribution, has been utilized to achieve the statistical consistency of either a classifier or the risk. Since the true transition matrix is unknown, previous researches have focused more on how to estimate this transition matrix well, and less on how to utilize it. In this paper, we propose a good utilization of the transition matrix is crucial for training a model, and suggest a new utilization method based on resampling, coined RENT. Specifically, we first demonstrate current utilizations can have potential limitations for implementation. As an extension to Reweighting, we suggest the Dirichlet distribution-based per-sample Weight Sampling (DWS) framework, and we compare reweighting and resampling under DWS framework. With the analyses from DWS, we propose RENT, a REsampling method with Noise Transition matrix. Empirically, RENT consistently outperforms existing transition matrix utilization methods, which includes reweighting, on various benchmark datasets.