PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Image Classification for Deep Representation Learning

Weijie Chen, Shiliang Pu, Di Xie, Shicai Yang, Yilu Guo, Luojun Lin

arXiv:2006.11480v2cs.CV

TL;DR

Deep clustering depends on storing global embeddings, which constrains its extension to extremely large datasets. UIC replaces embedding clustering with alternating pseudo-label generation and supervised-style representation learning. The authors report comparable performance with deep clustering and transfer-learning generalization.

  • Problem

    Embedding clustering requires storing the dataset’s global latent embeddings, limiting its extension to extremely large datasets.

    Source: Analysis · p. 5

  • Method

    UIC alternates image-wise pseudo-label generation from classifier softmax scores with supervised-style representation learning, without embedding clustering.

    Source: §1 · pp. 2–3

  • Results

    UIC is reported comparable with deep clustering and to surpass most self-supervised methods without performance degradation; its framework is twice faster than DeepCluster.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors present UIC as a simple prototype for developing more advanced unsupervised visual representation methods.

    Source: §1 · p. 3

Abstract

from arXiv · show

Deep clustering against self-supervised learning is a very important and promising direction for unsupervised visual representation learning since it requires little domain knowledge to design pretext tasks. However, the key component, embedding clustering, limits its extension to the extremely large-scale dataset due to its prerequisite to save the global latent embedding of the entire dataset. In this work, we aim to make this framework more simple and elegant without performance decline. We propose an unsupervised image classification framework without using embedding clustering, which is very similar to standard supervised training manner. For detailed interpretation, we further analyze its relation with deep clustering and contrastive learning. Extensive experiments on ImageNet dataset have been conducted to prove the effectiveness of our method. Furthermore, the experiments on transfer learning benchmarks have verified its generalization to other downstream tasks, including multi-label image classification, object detection, semantic segmentation and few-shot image classification.

1 Introduction

Embedding clustering makes deep clustering difficult to scale because it requires storing dataset-wide embeddings. The introduction presents UIC, which alternates pseudo-label generation and supervised-style representation learning without embedding clustering, and reports comparable performance with deep clustering.

1 Introduction

  • Embedding clustering requires saving each data point’s latent embedding, creating memory use that grows with dataset size and impeding very-large-scale training.The introduction motivates generating each image’s pseudo class ID without explicitly seeing other images.Source: §1 · p. 2
  • UIC predicts each image’s pseudo label with the highest softmax score, then learns representations using supervised-style training in alternating stages.Data augmentation is used both during representation learning and pseudo-label generation to perturb label assignment.Source: §1 · pp. 2–3
  • The paper interprets UIC as a special variant of both deep clustering and contrastive learning.It further reports that UIC’s classification results can act like clustering despite the lack of explicit clustering.Source: §1 · p. 3
  • The work proposes UIC as a simple framework for visual representation learning and as a prototype for developing more advanced unsupervised methods.The authors describe it as simplifying DeepCluster by discarding embedding clustering.Source: §1 · p. 3
  • UIC is reported to retain performance without degradation, surpass most other self-supervised methods, and run twice faster than DeepCluster by avoiding an extra forward pass.The method reuses prior-epoch forward results for pseudo-label generation.Source: §1 · p. 3

2 Related Work

The related work contrasts image-derived pretext tasks and multi-view invariance in self-supervised learning with clustering-based methods that learn features and pseudo-labels together. It positions UIC against the memory demands of embedding clustering and dataset-wide label optimization.

2.1 Self-supervised learning

  • Self-supervised methods use image-derived pretext tasks, including inpainting and jigsaw solving, while contrastive learning encourages similar features across views of the same image.The paper identifies multi-view invariant representations as the essence of its own method.Source: §2.1 · p. 3

2.2 Clustering-based methods

  • DeepCluster alternates k-means on dataset embeddings with end-to-end training, but storing all embeddings burdens scale; SelfLabel instead uses dataset-wide label optimization.The authors say UIC naturally produces nearly equal dataset partitions without label optimization.Source: §2.2 · p. 4

3 Methods

The method replaces global embedding clustering with classification-based pseudo-labeling and supervised-style representation learning. It frames this design in relation to deep clustering and contrastive learning.

3.1 Preliminary: Deep Clustering

  • Deep clustering assigns similar images to the same label through k-means on embeddings of all training images.The clustering objective jointly optimizes cluster centroids and assignments.Source: Pseudo Label Generation · p. 4

Pseudo Label Generation.

  • Representation learning trains a linear classifier and network together from pseudo-labels, alternating with label generation.The paper describes these stages as contributing positively to one another during optimization.Source: Pseudo Label Generation · p. 5

Analysis.

  • Storing the global embedding matrix scales with dataset size, while changing cluster IDs requires classifier reinitialization and can make the loss fluctuate.These issues motivate grouping images without explicitly using global relations.Source: Analysis · p. 5

3.2 Unsupervised Image Classification

  • UIC generates pseudo-labels by classifying images, then trains in a supervised manner using different random transformations for label generation and representation learning.Augmentation also perturbs pseudo-label generation; the classifier is optimized with the backbone, avoiding the global embedding matrix and classifier reinitialization.Source: §3.2 · pp. 5–6

The Relation with Embedding Clustering.

  • Classification differs from embedding clustering in using fixed orthonormal one-hot class centroids rather than dynamically updated centroids.UIC is described as a one-iteration variant with fixed centroids, while DeepCluster and DeeperCluster use 20 and 10 k-means iterations, respectively.Source: The Relation with Embedding Clustering · pp. 6–7

The Relation with Contrastive Learning.

4 Experimental Results

The experiments describe nearly uniform class assignments and evaluate ImageNet representations using linear probes and downstream transfer tasks. The section also identifies ablations on class number and augmentation during pseudo-label generation.

4.1 Dataset Benchmarks and Network Architectures

  • ImageNet experiments use 1.28 million images across 1,000 categories, with AlexNet as the baseline architecture.The AlexNet setup uses batch normalization in place of local response normalization.Source: §4.1 · p. 8
  • Evaluation includes linear probes and transfer learning on PASCAL VOC tasks plus metric-based few-shot classification on miniImageNet.The transfer tasks include multi-label classification, object detection, and semantic segmentation; few-shot evaluation avoids fine-tuning.Source: §4.1 · p. 8
  • The ablations examine class number and whether data augmentation is used during pseudo-label generation.Table 1 also reports NMI between pseudo-labels and annotated labels and defines FFT as further fine-tuning with fixed label assignments.Source: Table 1 · p. 9, Table 2 · p. 9
  • ImageNet images are divided into nearly equal class partitions by the classification model.The figure presents the number of images assigned to each class.Source: Figure 3 · p. 9

4.2 Unsupervised Image Classification

UIC uses class-balancing safeguards during training and otherwise follows supervised optimization settings, with class number as an additional hyperparameter.

Implementation Details.

  • Empty classes are split from the largest class at training’s start, while class-balance sampling helps avoid bias toward classes with more samples.The authors report that empty classes occur only at the beginning of training.Source: Implementation Details · p. 9

Optimization Settings.

  • AlexNet is trained for 500 epochs with SGD, Sobel-filtered inputs, random resized crops and horizontal flips; class number is the only extra hyperparameter versus supervised training.The class-number ablation tests 3k, 5k and 10k classes.Source: Optimization Settings · p. 9

Evaluation via Normalized Mutual Information.

UIC’s 10k-class assignments have NMI comparable to DeepCluster, and visualizations show nearly equal class partitions and semantically similar images grouped together. The section also describes linear-probe evaluation and reports that 3k classes slightly outperform 5k and 10k.

Evaluation via Normalized Mutual Information.

  • At 10k classes, NMI t/labels is comparable with DeepCluster, but this metric cannot directly compare different class counts because higher counts tend to raise it.Annotated labels are unavailable in practical scenarios and were not used to tune hyperparameters.Source: Evaluation via Normalized Mutual… · p. 10

Evaluation via Visualization.

  • UIC nearly divides dataset images into equal partitions, and its visualized classes contain images with similar semantic information.The authors infer class-balance sampling may implicitly bias assignments toward a uniform distribution, without label optimization.Source: Evaluation via Visualization · p. 10

4.3 Linear Classification on Activations

  • Linear probes evaluate frozen convolutional features by training inserted linear layers, with 10-crop accuracy measured after training.Activations are max-pooled to specified dimensions, and the feature extractors are frozen during linear-layer training.Source: §4.3 · pp. 10–11
  • 3k classes perform slightly better than 5k and 10k in the reported class-number ablation.The passage notes this result is also confirmed by.Source: §4.3 · p. 11

Ablation Study on Class Number Selection.

UIC further fine-tunes the network after fixing the final-epoch label assignments, and the reported performance improves.

Ablation Study on Class Number Selection.

  • After fixing last-epoch label assignments, 30 epochs of fine-tuning further improve performance.The fixed assignments use center-crop inference during pseudo-label generation.Source: Further Fine-Tuning · p. 11

Ablation Study on Data Augmentation.

Data augmentation during pseudo-label generation improves UIC’s performance. Across comparisons, UIC is comparable with DeepCluster, can surpass SelfLabel with strong augmentation, and surpasses most methods using a single supervisory signal.

Ablation Study on Data Augmentation.

  • Random resized cropping during pseudo-label generation improves performance, alongside its use during representation learning.The authors describe augmentation as important in clustering-based self-supervised learning and report the improvement in Tab.2.Source: Ablation Study on Data Augmentation · p. 11

Comparison with Other State-of-The-Art Methods.

  • UIC is comparable with DeepCluster and surpasses most self-supervised methods using a single supervisory signal; with strong augmentation, it also surpasses SelfLabel [3k × 1].The authors attribute the performance gap with SelfLabel to its extra augmentation and leave testing combinations with additional supervisory signals to future work.Source: Comparison with Other State-of-The-Art… · p. 12

4.4 Transfer to Downstream Tasks

UIC representations are evaluated on fine-tuned PASCAL VOC tasks and on miniImageNet few-shot classification without fine-tuning. The reported results are comparable with clustering-based methods, with an advantage over DeepCluster in the highest layers for few-shot classification.

Evaluation via Fine-Tuning: Multi-label Image Classification, Object Detection, Semantic Segmentation on Pascal-VOC

  • On PASCAL VOC, fine-tuned UIC is comparable with clustering-based methods and surpasses most other self-supervised learning methods.The evaluation covers multi-label image classification, object detection with Fast R-CNN, and semantic segmentation with FCN.Source: §4.4 · pp. 12–13

Evaluation without Fine-Tuning: Metric-based Few-shot Image Classification on miniImageNet

  • On miniImageNet, UIC is comparable with DeepCluster overall and performs better in the highest layers for metric-based few-shot classification without fine-tuning.The evaluation uses Prototypical Networks on the test set.Source: §4.4 · p. 13
  • The miniImageNet few-shot evaluation uses Prototypical Networks without fine-tuning, avoiding performance gaps from fine-tuning tricks.The table caption gives 68.2% as the supervised 5-way-5-shot accuracy reference.Source: §4.4 · p. 13, Table 5 · p. 13

5 More Experiments

The additional experiments test stronger data augmentation and a different network setup while retaining DeepCluster's training settings for fair comparison. Strong augmentation improves UIC's reported comparisons, and the ResNet50 setup reaches state-of-the-art results among non-contrastive methods while remaining behind SimCLR and MoCov2.

5 More Experiments

  • The experiments retain DeepCluster's training settings for fair comparison; achieving state-of-the-art results is not the work's primary aim.The authors note that further gains could be pursued by combining training tricks from other methods.Source: §5 · p. 13

5.1 More Data Augmentations

  • With strong augmentation and no further fine-tuning, UIC's conv5 result surpasses DeepCluster and SelfLabel by a large margin and is comparable with SelfLabel using 10 heads.The authors report these results in Table 6.Source: §5.1 · p. 13, Table 6 · p. 14

5.2 More Network architectures

  • With ResNet50 at 500 epochs, strong augmentation, and a fixed MLP head, UIC surpasses SelfLabel and achieves state-of-the-art results among non-contrastive methods, but remains behind SimCLR and MoCov2.The authors describe UIC as the simplest among these methods.Source: §5.2 · p. 14, Table 7 · p. 14

6 Conclusions

UIC's conclusions emphasize that embedding clustering is not the main reason DeepCluster works. The framework makes self-supervised training resemble supervised image classification and is proposed as a prototype for further method development.

6 Conclusions

  • UIC's results support the conclusion that embedding clustering is not the main reason DeepCluster works.Source: §6 · p. 14
  • UIC makes training a self-supervised model as easy as training a supervised image-classification model.Source: §6 · p. 14
  • The authors present UIC as a strong prototype for developing more advanced unsupervised learning approaches.Source: §6 · p. 14
Loading 2006.11480v2…