Source-linked AI summary
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
Mang Ye, Xu Zhang, Pong C. Yuen, Shih-Fu Chang
TL;DR
Unsupervised embedding learning needs similarity-preserving low-dimensional features without human labels. The paper uses instance-wise supervision in an instance feature-based softmax embedding trained with a Siamese strategy, reporting faster learning, higher accuracy, and strong performance across seen and unseen categories. The method also remains competitive without a pre-trained network on fine-grained categories.
Problem
Unsupervised embedding learning seeks discriminative low-dimensional features without human-annotated labels while preserving similarity between visual or category-related samples.
Method
The method directly optimizes real instance features with softmax, using augmentation invariance and instance separation through Siamese training.
Results
The method achieves significantly faster learning and higher accuracy than competing methods, with strong performance on seen and unseen testing categories and fine-grained samples without pre-training.
Takeaways & Limitations
Data augmentation invariance and instance spread-out properties help capture apparent visual similarity and generalize to unseen testing categories.
Takeaways & Limitations
The memory-bank alternative is inefficient because memorized features are updated only once per epoch while the network updates every iteration.
Abstract
from arXiv · showhide
This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based softmax embedding method, which directly optimizes the `real' instance features on top of the softmax function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.
1. Introduction
The paper addresses unsupervised embedding learning by using instance-wise relationships to learn augmentation-invariant and instance-separated features without category labels. It introduces an instance feature-based softmax method and reports faster learning, higher accuracy, and strong generalization across testing settings.
- Motivation: Unsupervised embedding learning seeks low-dimensional features that concentrate visually or categorically similar samples and separate dissimilar samples without human-annotated labels.Supervised methods target these properties but require costly annotation and domain expertise.
- Motivation: Instance-wise supervision approximates positive concentration and negative separation using relationships that do not depend on predefined categories.This supports generalization to categories not seen during training.
- Proposed idea: Random data augmentation supplies positive pairs by requiring features of the same image instance to remain invariant across transformations.The method uses augmentation because reliable positive mining is infeasible with a randomly initialized network.
- Proposed idea: The proposed instance feature-based softmax embedding directly optimizes inner products between instance features, while softmax mines hard negatives and uses relationships among sampled instances.A Siamese training strategy addresses the much larger number of instances than categories.
- Results: The method achieves significantly faster learning and higher accuracy than competing methods while learning data augmentation-invariant and instance spread-out features.These are reported as the paper’s primary method-level contributions.
- Results: The authors report that both invariant and spread-out properties capture apparent visual similarity, generalize to unseen categories, and achieve state-of-the-art performance across classification and embedding experiments.The evaluation covers comprehensive image classification and embedding learning experiments.
2. Related Work
Related work distinguishes unsupervised embedding learning from general unsupervised feature learning and reviews supervised embedding and unsupervised embedding approaches. The section emphasizes visual similarity, sampling relationships, and the challenge of unseen testing categories.
- General Unsupervised Feature Learning: General unsupervised feature learning typically learns an intermediate representation for later task-specific fine-tuning, whereas unsupervised embedding learning must also preserve visual similarity.The distinction concerns the properties required of the learned representation.
- Deep Embedding Learning: Deep embedding methods commonly minimize intra-class variation and maximize inter-class variation using pairwise or triplet relationships.Hard, semihard, and smart mining are sampling strategies used to improve performance.
- Unsupervised Embedding Learning: Unsupervised embedding evaluations distinguish seen testing categories from more challenging unseen testing categories that do not overlap with training categories.The unseen-category setting tests transfer beyond the categories used during training.
- Unsupervised Embedding Learning: Prior unsupervised embedding work mines hard positive and negative samples on manifolds, while this area addresses embedding without category-wise labels.The related-work discussion frames sampling as a central approach to discovering useful relationships.
3. Proposed Method
The method learns unsupervised embeddings by making augmented views of each instance invariant while spreading different instances apart. It directly optimizes real instance features with a softmax objective and uses Siamese-network mini-batch training to reduce computation.
- 3. Proposed Method: The embedding objective makes features of visually similar images close and features of dissimilar instances separated.Features are low-dimensional and ℓ2-normalized.
- 3. Proposed Method: Instance-wise supervision approximates category-level positive concentration and negative separation without category labels.Different augmentations of one instance provide positive pairs, while different instances provide spread-out negatives.
- 3.1. Instance-wise Softmax Embedding: Existing softmax embedding uses classifier weights or memorized features, limiting explicit feature comparison, efficiency, or discriminability.Memory-bank features are updated only when their instances are sampled, becoming outdated relative to the network.
- 3.2. Softmax Embedding on ‘Real’ Instance Feature: Directly replacing classifier weights or memory vectors with current features is problematic because self-similarity is always 1 and full-dataset feature computation is impractical.The proposed method addresses these issues by using original and augmented images together with a small randomly selected batch.
- 3.4. Training with Siamese Network: The Siamese network feeds randomly selected instances and their augmented views into two branches, using one positive and 2N −2 negatives per sample.This training strategy greatly reduces computational cost while supporting augmentation invariance and instance spread-out features.
4. Experimental Results
Experiments evaluate the proposed embedding on seen and unseen categories, using kNN and linear classification, retrieval, ablations, and training from scratch. The method achieves strong accuracy, faster learning, generalization to unseen categories, and sensitivity to augmentation and hard-negative sampling.
- CIFAR-10: 83.6% kNN accuracy is achieved on CIFAR-10, outperforming Exemplar CNN by 9.1%, NPSoftmax by 2.8%, and NCE by 3.2%.The authors attribute the improvement to directly optimizing the feature itself and to softmax hard-negative mining.
- CIFAR-10: 2 epochs are needed to reach 60% kNN accuracy, compared with 25 epochs for NPSoftmax and 45 epochs for Exemplar CNN.The paper links this efficiency to optimizing instance features rather than classifier weights or a memory bank.
- STL-10: 74.1% kNN and 69.5% linear accuracy are obtained on STL-10 with 5K training images, while full 105K-image training raises them to 81.6% and 77.9%.The proposed method is reported as best under both classifiers, and the results improve with more training samples.
- Unseen Testing Categories: Training from scratch on Product still makes the proposed method the clear winner, while MOM fails because randomly initialized features provide limited information for label mining.This experiment uses ResNet18 without pre-training.
- Ablation Study: Removing data augmentation reduces accuracy from 83.6% to 37.4%, while using easy negatives causes a dramatic drop and hard negatives nearly preserve full-model performance.The ablations support the roles of augmentation invariance and separating hard negative instances.
5. Conclusion
The paper addresses unsupervised embedding learning by learning data-augmentation-invariant and instance-spread-out features. A Siamese-based instance feature softmax approach directly optimizes instance features, yielding reported performance and efficiency gains.
- The method learns features invariant to data augmentation and spread out across instances.
- Figure 6 compares cosine-similarity distributions for randomly initialized and learned models across different attributes on CIFAR-10.
- A Siamese network pulls features of augmented views of the same instance together and pushes features of different instances apart.
- Directly optimizing instance features with the proposed softmax embedding produces significant performance and efficiency gains.