Deep Learning from Noisy Image Labels with Quality Embedding

I. INTRODUCTION

• Existing deep-learning approaches either use heuristic robust losses or latent labels, but do not explicitly model noisy-label trustworthiness.Latent-label transitions can leave corruption affecting predictions.Source: §I · p. 1
• Quality embedding introduces a variable for noisy-label trustworthiness, separating noisy-label mismatch from latent-label estimation error during back-propagation.The model aims to reduce noise effects while allowing supervision from high-quality labels to train the classifier.Source: §I · pp. 1–2, Figure 1 · p. 2
• CAN’s contrastive layer estimates quality in the embedding space, while its additive layer combines prior predictions and noisy labels as the posterior for classifier training.The contrastive layer reduces noise effects.Source: §I · p. 2
• Reparameterization tricks yield an efficient SGD algorithm intended to make the model scalable to big data.The paper also reports qualitative analyses of quality embeddings, latent-label estimates, and noise patterns.Source: §I · p. 2

II. RELATED WORK

• Robust-loss methods reweight, bootstrap, or rectify labels during training; the paper distinguishes its approach by additionally modeling noisy-label trustworthiness.Examples include sample-weighted cross-entropy, positive-sample reweighting, bootstrap, soft-label reweighting [28], and correction using extra clean data.Source: §II · pp. 2–3
• Latent-label methods adapt clean labels to noisy labels, but transitions may omit image content or rely on assumptions or clean data that constrain their use.The reviewed approaches include symmetric-noise assumptions [20], linear adaptation,, and methods using clean data,.Source: §II · p. 3

A. Preliminaries

• The task is multi-label, multi-class classification from noisy image-label pairs, with an underlying clean label and a D-dimensional Gaussian-space variable representing annotation quality.Images may be original images or extracted feature vectors; noisy labels are K-dimensional binary vectors indicating annotated categories.Source: A. Preliminaries · p. 3
• The quality-embedding graphical model marks image X and noisy label Y as observed, while latent label Z and quality variable S are latent; solid and dashed lines denote generation and inference.Source: Figure 2 · p. 3

B. Quality Embedding Model

• The generative model uses P(Z|X) for latent labels and P(Y|S,Z) for noisy labels; inference approximates both latent distributions from X and Y.Source: 1) quality embedding · p. 3
• Because the log-likelihood is difficult to compute explicitly, the model optimizes an evidence lower bound using variational distributions q(z_m|x_m,y_m) and q(s_m|x_m,y_m).The bound is presented as an approximation to the marginal likelihood and a basis for model selection.Source: 1) quality embedding · p. 3

2) variational mutual information regularizer:

• Mutual-information regularization addresses the possibility that flexible neural-network distributions prevent ELBO optimization from reaching a desirable solution.The regularizer minimizes the entropy of latent labels and quality variables, pushing latent-label posteriors toward extreme points and encouraging low variance for quality posteriors.Source: 2) variational mutual information… · p. 4

3) objective:

• The objective combines the ELBO with a weighted mutual-information regularizer, with coefficient λ controlling its effect in the minimization.Source: 3) objective · p. 4
• Unlike prior methods, the model conditions noisy-label transitions on both latent labels and quality, implements them nonlinearly, and approximates both posteriors without requiring extra clean data.The paper contrasts these choices with transitions based only on latent labels, linear transitions, and methods that may require clean data or other label knowledge.Source: 3) objective · p. 4

1) architectures:

• CAN uses end-to-end encoder, sampler, decoder, and classifier modules to infer labels and quality, sample latent variables, reconstruct noisy labels, and classify.The classifier is trained using KL divergence.Source: 1) architectures · pp. 4–5, Figure 3 · p. 5

2) contrastive layer and additive layer:

• The contrastive layer embeds the difference between noisy labels and prior predictions to estimate label quality and reduce noise effects.It maps the difference to the mean and log variance of the quality-variable distribution.Source: 2) contrastive layer and additive layer · p. 5
• The additive layer combines noisy labels and prior predictions nonlinearly to learn posterior labels for classifier training.Unlike manually tuned weighted combinations, this combination is learned by a neural network.Source: 2) contrastive layer and additive layer · p. 5
• The two branches jointly reduce noise effects and estimate posterior labels during end-to-end training.The quality-variable branch addresses noise while the additive layer estimates the posterior label.Source: 2) contrastive layer and additive layer · p. 5

• Monte Carlo estimates are needed because the objective term has no closed form when the variational distributions are not conjugate to the noisy-label distribution.Neural networks are used to model these distributions in the paper.Source: D. Optimization · p. 5
• Direct sampling for gradients of the variational distributions can have high variance, requiring many samples and increasing GPU load and computational burden.The section presents reparameterization tricks as a way to address this difficulty.Source: D. Optimization · p. 5
• Reparameterization separates parameter-related terms from parameter-free variates so Monte Carlo sampling has lower variance.The passage describes applying the approach to discrete z_m and continuous s_m.Source: D. Optimization · p. 5

1) The reparameterization tricks:

• The authors use Gumbel and Gaussian parameter-free variates in the reparameterized sampling procedure.The section applies the tricks to discrete z_m and continuous s_m.Source: 1) The reparameterization tricks · pp. 5–6
• Reparameterized Monte Carlo sampling yields a low-variance estimate of the objective term, and the authors report efficient estimation even with N = 1 sample per image.N counts the variates sampled for each image.Source: 1) The reparameterization tricks · p. 6

2) Stochastic variational gradient:

• The stochastic-gradient procedure makes the objective derivable with respect to all distribution parameters and supports learning them with SGD.The passage identifies this as important for deep learning on large datasets.Source: 2) Stochastic variational gradient · p. 6
• An annealed gradient addresses difficulty separating clean-label information from quality information and possible misalignment between latent and noisy label orders.The passage notes that a category in one dimension of z_m may correspond to a different dimension of y_m.Source: 2) Stochastic variational gradient · p. 6
• The gradient shifts from noisy-label cross-entropy initially toward the model objective as training proceeds.The schedule uses a time-varying weight that decays with training time.Source: 2) Stochastic variational gradient · p. 6
• Unlike a symmetric autoencoder, the model uses X and Y to infer Z and S, decodes them to Y, and jointly optimizes a discriminative X-to-Z model.In Fig. 4, solid and dashed lines denote decoding and encoding, respectively.Source: Figure 4 · p. 6, 2) Stochastic variational gradient · p. 6

IV. EXPERIMENTS

• Experiments evaluate CAN quantitatively and qualitatively, including comparisons with other methods and analyses of training size, hyperparameters, and model behavior.The experiments cover classification on noisy datasets and examine quality embeddings, latent-label estimation, and noise transitions.Source: §IV · p. 6

A. Datasets

• WEB contains 97,836 images from 20 PASCAL VOC categories, with 4,000–8,000 samples per category.Most images belong to one class; about 10,000 have two or more, and labels may contain annotation errors.Source: A. Datasets · pp. 6–7
• Fig. 5 shows the number of instances in each category for WEB and AMT, with WEB on the left and AMT on the right.Source: Figure 5 · p. 7
• AMT contains 7,354 annotated images across four dog breeds, with 1,000–2,000 samples per category and possible annotation errors.The images come from Stanford Dog and were annotated by Turkers.Source: A. Datasets · p. 7
• V07 and V12 provide clean 20-category classification data, while SD4 is a clean four-category dog dataset.The described test sets are V07TE, V12TE, and a randomly partitioned SD4TE; SD4 has 837 samples in total.Source: A. Datasets · p. 7

B. Experimental Setup

• WEB, V07, and V12 use a 34-layer residual network; AMT and SD4 use provided features with a three-layer perceptron.Images are resized and transformed for training, while testing averages predictions over six crops.Source: B. Experimental Setup · p. 7
• Training runs for 90 epochs with batch size 50, an initial learning rate of 0.01, and tenfold learning-rate reductions every 30 epochs.The regularizer coefficient is 0.3, and the sampler uses N=1.Source: B. Experimental Setup · p. 7
• Average Precision and mean Average Precision are the evaluation metrics.Source: B. Experimental Setup · p. 7
• Models trained on WEB and AMT are tested on V07TE, V12TE, and SD4TE as category compatibility allows; artificial-noise experiments train on clean training splits.Fig. 6 reports results across five training-size ratios for those test datasets.Source: B. Experimental Setup · p. 7, Figure 6 · p. 8

1) Training with real-world noisy datasets:

• 83.8% mAP on V07TE: CAN exceeds Resnet-N by 4.3% mAP and Bootstrap by 1.9% mAP, and outperforms baselines on the reported datasets.On SD4TE, CAN exceeds baselines by at least 2.8% mAP.Source: 1) Training with real-world noisy… · p. 8

2) Impact of training size:

• 81.0% mAP on V07TE with 20% of WEB training data: CAN remains ahead, while ICNM and LearnQ fall below Resnet-N at 79.4% mAP.All methods decline as training size decreases; the decline is more pronounced on SD4TE than V07TE and V12TE.Source: 2) Impact of training size · p. 8

3) hyperparameter sensitivity:

• 85.2% mAP on V12TE at λ=0.2; performance across datasets peaks and then declines as λ increases, falling below 76.6% mAP at λ=10.The authors report that setting λ between 0 and 1 makes the variational mutual information regularizer work with the KL-divergences.Source: 3) hyperparameter sensitivity · pp. 8–9, Table V · p. 8

4) controlled experiments with artificial noise:

• CAN outperforms the other baselines when artificial label-corruption probability is 0.8, 0.6, 0.4, or 0.2, but noise-aware models trail plain networks when the data are clean.At PNoise=1.0, all models perform close to randomness; results improve as corruption decreases from 1.0 to 0.Source: 4) controlled experiments with… · p. 9

1) quality embedding:

• Two distinguishable embedding clusters per category indicate that CAN identifies mismatches between latent and noisy labels; the reported embeddings are less clear for categories with fewer and less diverse training samples.The comparison includes two categories each from WEB and AMT.Source: 1) quality embedding · p. 9

D. Model Visualization

• Transition patterns conditioned on trustworthy embeddings concentrate on the diagonal, while those conditioned on non-trustworthy embeddings diffuse away from it.Source: Figure 9 · p. 10

2) latent label estimation:

• The additive-layer estimates rectify examples of incorrect WEB and AMT annotations, including unrelated WEB tags and wrong fine-grained dog labels.The authors connect training on these estimated latent labels with CAN's better performance than the baselines.Source: 2) latent label estimation · pp. 9–10, Figure 8 · p. 10

3) noise transition:

• Trustworthy embeddings show dominant diagonal transitions, while non-trustworthy embeddings show diffuse transitions on WEB and similar patterns on AMT.The authors interpret this as CAN using quality embeddings to disturb latent labels toward noisy labels.Source: 3) noise transition · p. 10
• Non-trustworthy transitions reflect dataset-specific noise, including “plt” annotations, “prs”/“tv” confusion, and transitions between similar dog breeds.The authors note Norfolk/Norwich Terrier and Irish/Scottish Wolfhound transitions in AMT, reflecting difficulty distinguishing those breeds.Source: 3) noise transition · pp. 10–11

V. CONCLUSION

• The conclusion reports that CAN outperforms other noise-aware deep-learning methods on some noisy training datasets, but validation covers image data only.The authors identify other content types as a direction for further exploration.Source: §V · p. 11
• The appendix computes latent-label KL terms by enumerating multinomial dimensions and quality-variable terms using a Gaussian prior and conjugation.The quality variable is modeled as a Gaussian with parameters produced by a network from the image and noisy label.Source: §V · p. 11