PaperlayerView exact revision on arXiv

Source-linked AI summary

ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learning

Hyuck Lee, Seungjae Shin, Heeyoung Kim

arXiv:2110.10368v1cs.LGcs.CV

TL;DR

Class imbalance can bias SSL classifiers and their unlabeled-data predictions, while labeled-data imbalance methods do not readily combine with SSL. The paper adds an auxiliary balanced classifier and class-balanced consistency regularization to an SSL backbone, and reports that the approach outperforms baseline algorithms across varied settings. Its stated setting assumes labeled and unlabeled data share the same class distribution.

  • Problem

    Most SSL algorithms assume balanced datasets, although real-world data may be imbalanced, and labeled-data class-imbalance methods do not readily combine with SSL.

    Source: §1 · p. 1

  • Method

    A single-layer auxiliary balanced classifier uses an SSL backbone's representations, class-balanced minibatch training, and modified consistency regularization on unlabeled data.

    Source: §1 · pp. 1–2

  • Results

    The proposed algorithm outperforms baseline algorithms in experiments under various class-imbalanced settings.

    Source: §5 · p. 10

  • Takeaways & Limitations

    The experiments support the approach as a class-imbalanced SSL method that uses backbone representations while training ABC to make class-balanced predictions.

    Source: §5 · p. 10

    The method assumes that labeled and unlabeled data are class-imbalanced to the same extent.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Existing semi-supervised learning (SSL) algorithms typically assume class-balanced datasets, although the class distributions of many real-world datasets are imbalanced. In general, classifiers trained on a class-imbalanced dataset are biased toward the majority classes. This issue becomes more problematic for SSL algorithms because they utilize the biased prediction of unlabeled data for training. However, traditional class-imbalanced learning techniques, which are designed for labeled data, cannot be readily combined with SSL algorithms. We propose a scalable class-imbalanced SSL algorithm that can effectively use unlabeled data, while mitigating class imbalance by introducing an auxiliary balanced classifier (ABC) of a single layer, which is attached to a representation layer of an existing SSL algorithm. The ABC is trained with a class-balanced loss of a minibatch, while using high-quality representations learned from all data points in the minibatch using the backbone SSL algorithm to avoid overfitting and information loss.Moreover, we use consistency regularization, a recent SSL technique for utilizing unlabeled data in a modified way, to train the ABC to be balanced among the classes by selecting unlabeled data with the same probability for each class. The proposed algorithm achieves state-of-the-art performance in various class-imbalanced SSL experiments using four benchmark datasets.

1 Introduction

Most SSL algorithms assume balanced classes, but real-world imbalance can bias classifiers and their pseudo-labels, while labeled-data imbalance methods do not readily transfer to CISSL. The paper adds a single-layer auxiliary balanced classifier (ABC) and modified consistency regularization to an SSL backbone, reporting state-of-the-art performance across varied CISSL experiments.

1 Introduction

  • Class-imbalanced SSL remains underexplored, and conventional class-imbalance techniques designed for labeled data cannot simply be combined with SSL.The introduction notes that SSL predictions on unlabeled data can become even more imbalanced; on CIFAR-10-LT, Class 0 has 100 times more examples than Class 9.Source: §1 · p. 1
  • A single-layer auxiliary balanced classifier is attached to an existing SSL backbone to mitigate imbalance while using unlabeled data.The ABC attaches to the representation layer immediately before the backbone's classification layer.Source: §1 · pp. 1–2
  • A stochastic mask forms a class-balanced minibatch subset for ABC training while retaining representations learned from all minibatch data.The authors describe this design as addressing prior resampling drawbacks of minority-class overfitting and majority-class information loss.Source: §1 · p. 2
  • The proposed algorithm achieves state-of-the-art performance across various class-imbalanced SSL experiments.The introduction also reports that its predicted-label distribution is more balanced than ReMixMatch's on the illustrated dataset.Source: §1 · p. 2
  • Modified consistency regularization selects unlabeled examples with equal probability across classes to encourage balanced ABC predictions.Consistency regularization keeps outputs similar across augmented or perturbed versions of an unlabeled example.Source: §1 · p. 2

2 Related Work

Related work covers SSL methods for unlabeled data, labeled-data class-imbalance techniques, and a small set of CISSL approaches. The paper builds on findings that representations can remain meaningful despite imbalance and that prior CISSL methods address biased pseudo-labels or minority-class data differently.

2 Related Work

  • SSL techniques include entropy minimization, mixup regularization, and consistency regularization; FixMatch and ReMixMatch use strong augmentation.FixMatch and ReMixMatch serve as backbones for the proposed algorithm.Source: §2 · p. 2
  • Class-imbalance methods include resampling and reweighting, but may cause overfitting, information loss, or unstable gradients.Other approaches include effective reweighting, meta-learning-based reweighting, and new loss forms.Source: §2 · p. 2
  • A cited finding attributes biased classification mainly to the classification layer while showing that meaningful representations can be learned from imbalanced data.The paper uses that finding to motivate ABC training on representations learned with FixMatch and ReMixMatch.Source: §2 · p. 3
  • Prior CISSL work includes self-supervised and semi-supervised learning for decision boundaries, DARP pseudo-label refinement, and CReST's preferential use of minority-class pseudo-labels [34].Source: §2 · p. 3

3 Methodology

ABC adds a balanced classifier to an SSL backbone and trains it end-to-end using balanced labeled and unlabeled losses. The setup targets class-balanced testing despite imbalanced labeled and unlabeled data.

3.1 Problem setting

  • The training setup assumes labeled and unlabeled data share an imbalanced class distribution, while the target model should perform effectively on a class-balanced test set.Classes are ordered by labeled-data count; the larger and smaller halves are designated majority and minority classes.Source: §3.1 · p. 3

3.2 Backbone SSL algorithm

  • ABC attaches to the backbone’s representation layer, reusing its high-quality representations while replacing its potentially biased classifier.The backbone can be FixMatch or ReMixMatch, and the same weakly and strongly augmented data can be reused to reduce computational cost.Source: §3.2 · p. 3

3.3 ABC for class-imbalanced Semi-supervised learning

  • A 0/1 mask gives minority examples a higher chance of contributing to ABC’s labeled classification loss, balancing training without resampling the backbone’s minibatch.Because the backbone learns representations from all minibatch points, the masked ABC loss avoids representation-level overfitting or information loss associated with oversampling or undersampling.Source: §3.3 · p. 4, Figure 2 · p. 4

3.4 Consistency regularization for ABC

  • ABC consistency training uses weak-view predictions as soft pseudo-labels for two strong views, with a mask to balance which unlabeled examples contribute.Only predictions above confidence threshold τ are used; the unlabeled mask parameter is gradually decreased during training.Source: §3.4 · pp. 4–5

3.5 End-to-end training

  • End-to-end training sums the ABC classification and consistency losses with the backbone loss, but new data are classified using ABC alone.The paper contrasts this with decoupled classifier training after representation learning.Source: §3.5 · p. 5

4 Experiments

Experiments compare ABC with class-imbalanced SSL baselines across benchmark settings, then examine its computational cost, representations, predictions, and design choices. ABC performs strongly across the reported settings while using minibatch training and a small auxiliary classifier.

4.1 Experimental setup

  • Experiments varied long-tailed and step imbalance and labeled-data ratios on CIFAR-10, CIFAR-100, and SVHN, and also evaluated 7.5 million LSUN images.The main settings used γ = 100 and β = 20% for CIFAR-10 and SVHN, and γ = 20 and β = 40% for CIFAR-100.Source: §4.1 · pp. 5–6

4.2 Experimental results

  • The proposed algorithm achieved the highest overall performance under the main setting, with improved minority-class performance; ReMixMatch-based algorithms outperformed their FixMatch-based counterparts.The comparison also reports that BALMS had poor overall performance without unlabeled training data, while the proposed method outperformed DARP+cRT combinations.Source: §4.2 · pp. 6–7, Table 1 · p. 7
  • Across varied CIFAR-10 imbalance and labeled-data settings, the proposed algorithm achieved the highest overall accuracy and improved minority-class performance.Its advantage over DARP+cRT grew as labeled data decreased and imbalance increased; its gap over CReST+PDA tended to grow with imbalance.Source: §4.2 · p. 7, Table 2 · p. 7
  • Under step imbalance, the proposed algorithm performed best with a larger margin than in long-tailed settings; it also outperformed baselines on large-scale LSUN.CReST-based algorithms were excluded from LSUN comparisons because their repeated pseudo-label updates require loading the full unlabeled dataset.Source: §4.2 · pp. 7–8, Table 3 · p. 8, Table 4 · p. 8

4.3 Complexity of the proposed algorithm

  • ABC adds few parameters and shares the backbone representation layer, limiting added memory and training-time costs; minibatches also make large-scale LSUN training feasible.Its added parameters were 0.09% of the backbone for CIFAR-10 and 0.87% for CIFAR-100; DARP and CReST incurred additional computation or retraining costs.Source: §4.3 · p. 8

4.4 Qualitative analysis of high-quality representations and balanced classification

  • On CIFAR-10-LT, t-SNE showed that FixMatch+ABC and ReMixMatch+ABC learned more class-separable representations than ABC without an SSL backbone.ReMixMatch produced more separable representations than FixMatch in this example, indicating that backbone choice affects ABC performance.Source: §4.4 · p. 8, Figure 3 · p. 9
  • On CIFAR-10 test data, ReMixMatch+ABC classified minority classes more accurately and produced more balanced predictions than ReMixMatch+DARP+cRT.ReMixMatch often confused minority classes 8 and 9 with majority classes 0 and 1; DARP+cRT reduced but did not eliminate this pattern.Source: §4.4 · p. 9, Figure 4 · p. 9

4.5 Ablation study

  • Ablations associated overbalancing, majority bias, unstable training, or lower performance with removing key ABC controls or using decoupled training.The reported issues followed omitting the Bernoulli schedule, consistency regularization, masks, or confidence threshold; using hard pseudo-labels, replacing the mask with reweighting, or removing the backbone also hurt.Source: §4.5 · pp. 9–10, Table 5 · p. 10

5 Conclusion

The conclusion presents ABC as a balanced classifier attached to an SSL backbone and reports that the proposed algorithm outperforms baseline algorithms across various settings. It also states an assumption about the labeled and unlabeled class distributions and flags potential social risks.

5 Conclusion

  • Across various settings, the proposed algorithm outperformed baseline algorithms, with qualitative analysis and ablation study examining its components.The method assumes labeled and unlabeled data are imbalanced to the same extent; future work aims to estimate class distribution.Source: §5 · p. 10
  • The conclusion notes that imbalanced-data algorithms could favor majority groups and that ABC could also be misused to discriminate against minorities.Source: §5 · p. 10

A Overall procedure of consistency regularization for ABC

B Pseudo code of the proposed algorithm

C Two types of class imbalance for the considered datasets

The considered datasets include long-tailed and step imbalance, with step imbalance leaving very little data for each minority class. The supplied passage also specifies the settings used to illustrate both types.

D Specification of the confidence threshold τ

For the main setting, the proposed algorithm is reported to perform in a balanced way by G-mean. The appendix reports that τ = 0.95 gave the highest mean and lowest standard deviation in validation accuracy among the tested thresholds, alongside further training details.

D Specification of the confidence threshold τ

E Further details of the experimental setup

  • ReMixMatch training gradually increased the unlabeled-data loss coefficient following; FixMatch used labeled CIFAR-100 data without labels as additional unlabeled data, but not for CIFAR-10 or SVHN.The experiments otherwise followed default ReMixMatch and FixMatch settings unless stated otherwise.Source: E Further details of the experimental… · p. 14
  • For CIFAR-10 and SVHN under step imbalance, gradually decreasing B prevents unstable ABC training by using majority-class labeled examples more frequently.Source: E Further details of the experimental… · pp. 14–15

F Geometric mean (G-mean) of class-wise accuracy under the main setting

G Experimental results on SVHN and CIFAR-100 under various settings

The reported SVHN and CIFAR-100 results across various settings followed the trend described for CIFAR-10. One SVHN comparison could not be measured because the pseudo-label-refinement optimization did not converge.

H Experimental results on SVHN and CIFAR-100 under the step imbalanced setting

Under step imbalance, SVHN and CIFAR-100 results followed the same tendency as CIFAR-10. The FLOPS comparison reports that ABC’s minibatch training cost did not increase significantly, unlike DARP-based methods as classes or data grow.

H Experimental results on SVHN and CIFAR-100 under the step imbalanced setting

I Floating point operations per second (FLOPS) of each algorithm

J Further qualitative analysis and quantitative comparison

The qualitative comparisons report more balanced predictions with ABC and more separable representations than resampling-based CNNs. They also describe improved minority-class classification on test and unlabeled data.

K Further qualitative analysis and quantitative comparison

K Detailed comparison between the end-to-end training of the proposed algorithm and decoupled learning of representations and a classifier

ABC is trained end-to-end with the backbone’s representations and uses unlabeled data for consistency regularization, unlike decoupled DARP+cRT classifiers. The reported comparisons favor ABC variants over DARP+cRT, and their validation loss decreases steadily.

K Detailed comparison between the end-to-end training of the proposed algorithm and decoupled learning of representations and a classifier

  • FixMatch+ABC and ReMixMatch+ABC performed better than their DARP+cRT counterparts despite using the same backbone representations.The paper attributes the difference to ABC’s interactive end-to-end training and use of unlabeled data for consistency regularization after representation learning.Source: K Detailed comparison between the… · pp. 18–19, Table 14 · p. 19
  • Decoupled classifiers’ validation loss tended to rise after a few epochs, while consistency regularization reduced but did not eliminate that tendency.For variants that did not fix representations, backbone representations were gradually replaced, causing overfitting on minority classes.Source: K Detailed comparison between the… · p. 19, Figure 11 · p. 19
  • ReMixMatch+ABC, FixMatch+ABC, and the supervised end-to-end algorithm steadily reduced validation loss and achieved the lowest validation loss.The comparison is summarized in Table 14.Source: K Detailed comparison between the… · p. 19, Table 14 · p. 19

L Ablation study for FixMatch [29] + ABC on CIFAR-10

The FixMatch+ABC ablation results are reported to follow a similar tendency to the ReMixMatch+ABC results discussed in the main paper.

L Ablation study for FixMatch [29] + ABC on CIFAR-10

Loading 2110.10368v1…