Source-linked AI summary
SimMatch: Semi-supervised Learning with Similarity Matching
Mingkai Zheng, Shan You, Lang Huang, Fei Wang, Chen Qian, Chang Xu
TL;DR
Learning with few labeled examples motivates methods that can exploit unlabeled data more effectively. SimMatch jointly matches semantic and instance similarities, uses labeled examples to propagate pseudo-labels between them, and reports strong benchmark performance, including 67.2% and 74.4% Top-1 accuracy on ImageNet with 1% and 10% labels.
Problem
Few labeled data are expensive to collect, making effective use of unlabeled data an important semi-supervised learning challenge.
Method
SimMatch applies consistency regularization to semantic predictions and instance relationships, using a labeled memory buffer with unfolding and aggregation to propagate pseudo-labels.
Results
SimMatch improves performance across semi-supervised learning benchmarks and settings, achieving 67.2% and 74.4% ImageNet Top-1 accuracy with 1% and 10% labeled examples after 400 epochs.
Takeaways & Limitations
Semantic and instance similarities can be combined in SimMatch to generate higher-quality and more reliable matching targets.
Abstract
from arXiv · showhide
Learning with few labeled data has been a longstanding problem in the computer vision and machine learning research community. In this paper, we introduced a new semi-supervised learning framework, SimMatch, which simultaneously considers semantic similarity and instance similarity. In SimMatch, the consistency regularization will be applied on both semantic-level and instance-level. The different augmented views of the same instance are encouraged to have the same class prediction and similar similarity relationship respected to other instances. Next, we instantiated a labeled memory buffer to fully leverage the ground truth labels on instance-level and bridge the gaps between the semantic and instance similarities. Finally, we proposed the \textit{unfolding} and \textit{aggregation} operation which allows these two similarities be isomorphically transformed with each other. In this way, the semantic and instance pseudo-labels can be mutually propagated to generate more high-quality and reliable matching targets. Extensive experimental results demonstrate that SimMatch improves the performance of semi-supervised learning tasks across different benchmark datasets and different settings. Notably, with 400 epochs of training, SimMatch achieves 67.2\%, and 74.4\% Top-1 Accuracy with 1\% and 10\% labeled examples on ImageNet, which significantly outperforms the baseline methods and is better than previous semi-supervised learning frameworks. Code and pre-trained models are available at https://github.com/KyleZheng1997/simmatch.
1. Introduction
SimMatch addresses semi-supervised learning with few labels by matching semantic and instance similarities across augmented views. A labeled memory buffer and mutual pseudo-label propagation support reliable targets, yielding strong ImageNet results.
- Few labeled data remain expensive to collect, motivating semi-supervised learning methods that exploit large volumes of unlabeled data.
- Existing pseudo-labeling and consistency methods generally derive class predictions from weak or strongly augmented views and train against those targets.
- SimMatch simultaneously matches semantic similarity, represented by class predictions, and instance similarity across weakly and strongly augmented views.
- A labeled memory buffer uses ground-truth labels for instance-level matching and enables semantic and instance pseudo-labels to mutually propagate through aggregation and unfolding.
- 67.2% and 74.4% Top-1 accuracy are achieved with 1% and 10% labeled examples on ImageNet after 400 epochs of training.
2. Related Work
Related work spans consistency regularization, augmentation-anchored pseudo-labeling, self-supervised contrastive learning, and methods combining consistency with instance relationships. SimMatch builds on these directions while targeting limitations involving class collisions and hyperparameter sensitivity.
- Consistency regularization enforces similar predictions for different perturbed views of the same instance, using augmentations or network regularization.
- MixMatch, ReMixMatch, and FixMatch use augmentation consistency with pseudo-labels generated from augmented predictions, with methods adding sharpening, MixUp, or distribution alignment.
- Contrastive learning treats augmented views of one instance as positives and views from different instances as negatives, but similar instances can create class collisions.
- SimMatch generates semantic and instance pseudo-labels from class centers and labeled embeddings, then fuses them through unfolding and aggregation.
- Clustering and negative-free pretext tasks address class collision and have improved downstream classification performance.
- CoMatch combines consistency regularization with contrastive learning but is reported as sensitive to dataset- and setting-specific temperature and threshold choices.
3. Method
SimMatch combines semantic and instance-level consistency, using labeled memory and bidirectional label propagation to calibrate pseudo-labels across both similarity spaces.
- 3.2. Instance Similarity Matching: SimMatch enforces consistency between weakly and strongly augmented views at both semantic and instance levels.Semantic consistency matches class predictions, while instance consistency matches similarity distributions to other instances.
- 3.2. Instance Similarity Matching: Instance similarity is computed from L2-normalized embeddings, converted into a temperature-controlled softmax distribution, and matched across augmentations.A nonlinear projection head maps representations to low-dimensional embeddings before similarity computation.
- 3.3. Label Propagation through SimMatch: The labeled memory buffer assigns ground-truth classes to stored embeddings, turning instance references into class-linked information.The buffer stores individual class references alongside the classifier’s centered class references.
- 3.3. Label Propagation through SimMatch: Unfolding maps semantic similarities into the instance space by matching each stored embedding to its ground-truth class, then scales instance similarities with the unfolded values.This produces calibrated instance pseudo-labels that replace the original instance targets.
- 3.3. Label Propagation through SimMatch: Aggregation maps instance similarities into class space by summing similarities sharing ground-truth labels, then smooths semantic predictions with the aggregated distribution.The adjusted semantic pseudo-label replaces the original semantic target, enabling mutual propagation between the two spaces.
- 3.3. Label Propagation through SimMatch: When semantic and instance similarities agree, label propagation yields sharper, higher-confidence pseudo-labels; disagreement produces flatter pseudo-labels.The overall objective combines semantic and instance consistency losses with balancing factors λu and λin.
- 3.4. Efficient Memory Buffer: For large buffers, SimMatch uses a student-teacher encoder arrangement; for small buffers, temporal ensemble smoothing avoids maintaining a teacher network.The largest reported buffer stores approximately 10^5 embeddings and requires 64M GPU memory for features.
4. Experiments
SimMatch is evaluated across CIFAR, ImageNet, transfer-learning, efficiency, pseudo-label, and ablation settings using standard semi-supervised benchmarks and controlled component comparisons. It generally achieves strong results, with gains especially evident under limited labels and in ImageNet efficiency comparisons.
- Datasets and settings: Experiments evaluate SimMatch on CIFAR-10, CIFAR-100, and ImageNet-1k with varying labeled-data settings.CIFAR experiments use 4, 25, and 400 labels for CIFAR-10 and 4, 25, and 100 labels for CIFAR-100; ImageNet uses 1% and 10% settings.
- Benchmark results: SimMatch achieves state-of-the-art performance across the reported CIFAR settings, with its largest CIFAR-10 gain occurring with 40 labels.Improvements on CIFAR-10 are relatively small with 250 and 4000 labels, which the authors associate with accuracy already nearing supervised performance.
- Benchmark results: 67.2% and 74.4% Top-1 accuracy are achieved on ImageNet with 1% and 10% labeled examples after 400 training epochs.The reported 74.4% result is slightly above FixMatch-EMAN's 74.0% under the 10% setting.
- Transfer learning: SimMatch produces the best reported transfer-learning performance on CIFAR-10, CIFAR-100, Cars, and Flowers using ImageNet-pretrained representations.The results are comparable with BYOL and significantly better than SimCLR, MoCo V2, and the supervised baseline.
- Training efficiency: SimMatch is nearly 17% faster than FixMatch and CoMatch in measured GPU hours per epoch.The efficiency difference is attributed to avoiding a retained computational graph for weakly augmented data and requiring one rather than two EMA-network passes compared with CoMatch.
5. Conclusion
SimMatch combines semantic- and instance-level consistency regularization, adds a labeled memory buffer, and enables information propagation between both similarity levels. Experiments report effective components and state-of-the-art ImageNet semi-supervised performance.
- SimMatch applies consistency regularization at both semantic and instance levels.
- A labeled memory buffer leverages data annotations at the instance level.
- Unfolding and aggregation propagate labels between semantic-level and instance-level information.
- The experiments show the effectiveness of each SimMatch component.
- Results on ImageNet-1K demonstrate state-of-the-art semi-supervised learning performance.