Source-linked AI summary
CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise
Kuang-Huei Lee, Xiaodong He, Lei Zhang, Linjun Yang
TL;DR
Learning image classifiers from noisy labels requires balancing costly human verification against less effective supervision-free methods. CleanNet transfers label-cleaning knowledge from verified classes through joint embeddings and jointly trains with a classifier, improving noise detection and noisy-data classification. Its scope remains constrained by the scalability of verifying correct classes and by the assumption that dataset labels may be incorrect.
Problem
Existing human-supervised label-noise methods are costly and hard to scale across classes, while methods without human supervision are less effective.
Method
CleanNet learns class and query embeddings from verified reference images, transfers label-correctness knowledge to other classes, and jointly trains with an image classifier.
Results
CleanNet improves label-noise detection on held-out classes and significantly improves image-classification accuracy on noisy data.
Takeaways & Limitations
CleanNet can make label-noise detection and learning from noisy data with human supervision more scalable through transfer learning.
Takeaways & Limitations
Verifying correct classes, as in Clothing1M, is not scalable when workers must select from many classes, and the setup assumes some labels are incorrect.
Abstract
from arXiv · showhide
In this paper, we study the problem of learning image classification models with label noise. Existing approaches depending on human supervision are generally not scalable as manually identifying correct or incorrect labels is time-consuming, whereas approaches not relying on human supervision are scalable but less effective. To reduce the amount of human supervision for label noise cleaning, we introduce CleanNet, a joint neural embedding network, which only requires a fraction of the classes being manually verified to provide the knowledge of label noise that can be transferred to other classes. We further integrate CleanNet and conventional convolutional neural network classifier into one framework for image classification learning. We demonstrate the effectiveness of the proposed algorithm on both of the label noise detection task and the image classification on noisy data task on several large-scale datasets. Experimental results show that CleanNet can reduce label noise detection error rate on held-out classes where no human supervision available by 41.5% compared to current weakly supervised methods. It also achieves 47% of the performance gain of verifying all images with only 3.2% images verified on an image classification task. Source code and dataset will be available at kuanghuei.github.io/CleanNetProject.
1. Introduction
Large-scale recognition relies on noisy Internet-labeled data because manual labeling is costly, but label noise harms classifier accuracy. CleanNet transfers label-cleaning knowledge from verified classes to unverified classes and jointly supports noise detection and classifier training.
- Motivation: Manual supervision scales poorly because every class requires labeling effort, while unsupervised approaches are more scalable but less effective.Internet-collected noisy labels are a practical substitute for expensive, time-consuming manual annotation.
- CleanNet: CleanNet learns representative class prototypes from supervised reference images and transfers that knowledge to other classes without explicit human supervision.An attention mechanism selects representative seed images and encodes each class into an embedding.
- CleanNet: CleanNet compares query and class embeddings to identify whether an image is mislabeled relative to its noisy class label.Relevant queries should have embeddings similar to their class embeddings.
- Joint learning: CleanNet assigns image weights from image-to-label relevance, while a unified learning scheme jointly trains CleanNet and the image classifier.The classifier is refreshed to provide improved convolutional features for CleanNet.
- Evaluation: The study evaluates label-noise detection and image classification on large datasets including Clothing1M, WebVision, and Food-101N.Food-101N contains 310K Internet-collected images with verification labels.
2. Related Work
Prior label-noise methods include outlier removal, human verification, specialized data sources, knowledge graphs, and neural transfer learning. Their scalability or generalization can be limited by assumptions, class-by-class verification, or dependence on specific resources.
- Label noise reduction: Unsupervised outlier-removal methods rely on the assumption that outliers are mislabeled, although outliers are often not well defined.This makes deciding which instances to remove challenging.
- Supervised cleaning: Human-verification approaches can require expensive verification across classes and may not generalize to classes that were not manually verified.Some alternatives depend on specific data sources or knowledge graphs, limiting generalization beyond their intended settings.
- Transfer learning: Neural joint-embedding methods use supervised and unsupervised objectives to support transfer learning and robustness.Related work also includes adversarial objectives for domain adaptation.
3. Scalable Learning with Label Noise
CleanNet transfers label-noise knowledge from manually verified classes to unverified classes through joint reference-set and query embeddings. It then uses image-to-class relevance to weight noisy training samples while jointly improving the classifier and CleanNet.
- CleanNet: CleanNet learns from noisy labeled images with only a fraction of classes manually verified, transferring label-noise knowledge to other classes.The dataset contains n images with potentially incorrect class labels, while most verification labels can remain unavailable.
- CleanNet: The reference-set encoder maps representative feature vectors for a class into a class-level embedding using an attention mechanism.Features may be selected by random sampling or K-means; the experiments use 50 K-means feature vectors.
- CleanNet: The query encoder maps each image feature to an embedding, and cosine similarity with the class embedding indicates whether its noisy label is relevant.Manual verification labels train the matching constraint, maximizing similarity for relevant queries and minimizing it for mislabeled queries.
- CleanNet: CleanNet also pseudo-labels unverified queries when their query-to-class cosine similarity exceeds margin ρ, reinforcing similarity for relevant queries while ignoring others.The unsupervised objective complements the supervised matching loss, and both are combined in the total loss.
- CleanNet: Tied encoder parameters transfer information across classes, enabling label-noise detection without explicit human verification for every class.Images can be ranked by cosine similarity and filtered using a cross-validated threshold δ, usually shared across classes.
- CleanNet for Learning Classifiers: CleanNet assigns soft or hard attention weights to noisy samples, and these weights guide classifier training within a jointly learned CNN framework.Hard weighting is equivalent to explicit label-noise removal, while the classifier and CleanNet improve each other through alternating feature and weighting updates.
4. Experiments
Experiments evaluate CleanNet for label-noise detection and noisy-image classification across Food-101N, Clothing1M, and WebVision, including transfer-learning settings with limited verification labels. Results show that CleanNet improves detection and classification while transferring supervision to classes without explicit human verification.
- 4.1. Datasets: Food-101N contains 310K Internet images with approximately 80% noisy-label accuracy, plus manually added verification labels for training and testing detection.Clothing1M contains 1M noisy images across 14 classes, with an estimated label accuracy of 61.54%.
- 4.2. Label Noise Detection: CleanNet achieves 9.61% average label-noise detection error on Food-101N and 15.91% on Clothing1M, comparable to the supervised MLP baseline.The corresponding MLP errors are 10.42% and 16.09%, respectively; classification filtering is worse on both datasets.
- 4.2. Label Noise Detection: 41.5% relative error reduction is achieved on held-out Food-101N classes at n = 10 compared with classification filtering.With 50/101 classes held out, CleanNet reaches 11.02% error, comparable to the supervised MLP result of 10.12%.
- 4.3. Learning Classifiers with Label Noise: 2.1% and 1.75% accuracy gains remain when verification labels are withheld for 50/101 and 70/101 Food-101N classes, respectively.These held-out-class experiments transfer label-cleaning information from verified classes through CleanNet.
- 4.3. Learning Classifiers with Label Noise: 74.69% top-1 accuracy is obtained on Clothing1M with CleanNet, improving over 68.94% without label-noise handling.After fine-tuning on 50k clean images, the method reaches 79.90%, compared with 80.38% reported by a method using additional label-confusion information.
- 4.3. Learning Classifiers with Label Noise: 47% of the every-image verification performance gain is achieved on WebVision with only 3.20% of images verified.The every-image condition improves top-1 accuracy from 67.76% to 70.31%; semantic-308 and random-118 achieve 47% and 29% of that gain, respectively.
5. Conclusion
CleanNet addresses the tension between scalable supervision and effective label-noise handling by transferring verified label correctness across classes. It performs strongly with limited verification and can match extensively supervised methods when more classes are verified.
- CleanNet transfers supervised label-correctness information to classes without explicit human supervision.
- CleanNet outperforms methods using no human supervision by a large margin when a small fraction of classes is manually verified.
- CleanNet matches existing methods requiring extensive human supervision when sufficient classes are manually verified.