Source-linked AI summary
Unsupervised Deep Learning by Neighbourhood Discovery
Jiabo Huang, Qi Dong, Shaogang Gong, Xiatian Zhu
TL;DR
Exhaustive annotation requirements limit the scalability of supervised CNN training, motivating unsupervised learning of discriminative visual representations. The paper proposes AND, which progressively discovers class-consistent local neighbourhoods to learn latent class boundaries, and reports advantages over state-of-the-art unsupervised methods across six image benchmarks.
Problem
Supervised CNN training relies on exhaustive labels, limiting deployment and scalability in applications with restricted labelling resources.
Method
AND progressively discovers sample-anchored, class-consistent neighbourhoods and propagates their local relationships to learn discriminative representations without labels.
Results
AND shows performance advantages over a wide range of state-of-the-art unsupervised deep learning models across coarse- and fine-grained image classification benchmarks.
Takeaways & Limitations
Local neighbourhood supervision preserves clustering’s class-boundary inference while avoiding grouping noise and improving on sample-specificity learning.
Takeaways & Limitations
AND depends on neighbourhoods being class consistent, but the instance-based formulation provides no stable guarantee that all discovered neighbourhoods are reliable.
Abstract
from arXiv · showhide
Deep convolutional neural networks (CNNs) have demonstrated remarkable success in computer vision by supervisedly learning strong visual feature representations. However, training CNNs relies heavily on the availability of exhaustive training data annotations, limiting significantly their deployment and scalability in many application scenarios. In this work, we introduce a generic unsupervised deep learning approach to training deep models without the need for any manual label supervision. Specifically, we progressively discover sample anchored/centred neighbourhoods to reason and learn the underlying class decision boundaries iteratively and accumulatively. Every single neighbourhood is specially formulated so that all the member samples can share the same unseen class labels at high probability for facilitating the extraction of class discriminative feature representations during training. Experiments on image classification show the performance advantages of the proposed method over the state-of-the-art unsupervised learning models on six benchmarks including both coarse-grained and fine-grained object image categorisation.
1. Introduction
The paper introduces Anchor Neighbourhood Discovery (AND), an unsupervised method that progressively learns discriminative representations from locally class-consistent neighbourhoods without manual labels. Experiments report advantages over existing state-of-the-art unsupervised methods across coarse- and fine-grained image benchmarks.
- CNN progress in computer vision depends heavily on exhaustively labelled training imagery, restricting scalability and usability when labelling budgets are limited.
- AND progressively discovers sample-anchored neighbourhoods and uses them to learn latent class decision boundaries without manual supervision.The method combines local neighbourhood discovery with progressive curriculum learning.
- Each neighbourhood is designed to contain samples likely sharing an unseen class label, supporting extraction of class-discriminative feature representations.
- The method combines clustering and sample-specificity learning while adding sample-to-sample relationships through neighbourhood supervision.
- AND outperforms a wide variety of existing state-of-the-art unsupervised deep learning models on four coarse-grained and two fine-grained image classification datasets.The benchmarks include CIFAR10, CIFAR100, SVHN, ImageNet, CUB200-2011, and Stanford Dogs.
2. Related Work
Prior unsupervised approaches use clustering, sample specificity, self-supervision, or generative modelling, but each leaves important limitations in representing class structure. AND instead identifies local neighbourhoods to reduce clustering errors while retaining inter-sample class information.
- Unsupervised deep learning methods include clustering, sample specificity learning, self-supervised learning, and generative models.
- Clustering seeks class-consistent groups but becomes difficult on complex image data because its solution space grows exponentially with data and cluster size.
- AND replaces global clustering with local neighbourhood identification to control clustering errors and reduce their negative propagation.
- Sample specificity treats every sample as an independent class and does not explicitly model class decision boundaries.
- Self-supervised methods rely on hand-crafted auxiliary supervision, leaving the design of more target-related auxiliary supervision as an open problem.
- AND is distinct from constrained clustering because it uses no pairwise links, making the approach unsupervised and more scalable.
3. Unsupervised Neighbourhood Discovery
Anchor Neighbourhood Discovery (AND) learns unsupervised feature representations by discovering local, class-consistent neighbourhoods around individual samples and progressively using them for supervision. It combines neighbourhood discovery, differentiable neighbourhood supervision, and curriculum selection to avoid directly inferring global class boundaries.
- Neighbourhood Discovery: AND addresses unsupervised feature learning by estimating local class relationships in sample-anchored neighbourhoods rather than directly reasoning about global class boundaries.The method uses a divide-and-conquer perspective: discover local neighbourhoods, then propagate their inter-sample class relationships during training.
- Neighbourhood Discovery: The method begins with per-sample neighbourhoods, builds anchor neighbourhoods from learned features, and incorporates their structure through a differentiable neighbourhood supervision loss.This supports end-to-end optimisation while treating visually similar neighbourhood members as likely to share a class.
- Neighbourhood Discovery: AND assumes each anchor neighbourhood is sufficiently class consistent to assign its members a shared learning concept, while acknowledging that neighbourhood information is partial and distributed across multiple anchors.A neighbourhood may cover only part of a class, and several neighbourhoods may represent different aspects of the same underlying class.
- Neighbourhood Discovery: Unlike clustering, AND does not require complete class-boundary inference or prior knowledge of the cluster count; unlike sample specificity learning, it retains local class-level information.The design uses local neighbourhoods to reduce the impact of class inconsistency while preserving information about class relationships.
- Neighbourhood Selection: Because initial neighbourhoods are not reliably class consistent, AND progressively selects neighbourhoods through an easy-to-hard curriculum instead of relying on a one-off selection.The need for selection follows from the instability of neighbourhood reliability under the initial instance-based objective.
- Neighbourhood Selection: The curriculum uses entropy of each sample’s similarity-probability distribution as a consistency indicator, selecting lower-entropy neighbourhoods first for neighbourhood-supervised learning.Smaller entropy values are treated as indicating more consistent neighbourhoods, while unselected samples remain individual neighbourhoods.
4. Experiments
Experiments evaluate AND across six image-classification benchmarks, using unsupervised representations and downstream classification protocols. AND generally outperforms competing unsupervised methods, while analyses examine scalability, architecture, curriculum, neighbourhood size, and attention dynamics.
- Experimental setup: Evaluation uses six image-classification benchmarks, AlexNet by default, and no class labels during representation learning.Ground-truth labels are used only to enable test-time image categorisation.
- Comparisons to the State-of-the-Art Methods: AND often achieves the best results on three small-scale benchmarks, with larger margins for weighted kNN than linear regression.The comparison uses weighted kNN with FC features and linear regression with conv5 features.
- Comparisons to the State-of-the-Art Methods: On ImageNet, AND outperforms all competitors, although margins are smaller when using neighbourhoods of size 2.The authors suggest larger neighbourhoods may better capture structural information in large datasets.
- Component Analysis and Discussions: 41.2% with ResNet18 versus 31.3% with AlexNet on ImageNet shows that AND benefits from stronger backbone architectures.The study also tests network generalisation on CIFAR10.
- Component Analysis and Discussions: The smallest neighbourhood size, k = 1, performs best on CIFAR10, consistent with a preference for smaller neighbourhoods in varied imagery.Neighbourhood size controls the label consistency of anchor neighbourhoods.
- Component Analysis and Discussions: Four curriculum rounds provide a good efficiency–performance trade-off, while 100 epochs per round reaches 84.8% versus 83.3% with 50.More curriculum rounds increase training costs.
- Component Analysis and Discussions: During curriculum learning, class-consistent neighbourhoods increase faster than inconsistent ones, while attention can shift toward class-relevant image regions.The model may still fail to capture object attention under cluttered backgrounds or poor lighting.
5. Conclusion
The paper concludes that Anchor Neighbourhood Discovery (AND) learns discriminative deep models unsupervisedly through progressively discovering class-consistent neighbourhoods. It reports performance superiority over state-of-the-art unsupervised methods and examines learning dynamics and failure cases.
- Conclusion: AND enables unsupervised learning of discriminative deep network models through progressively discovered class-consistent neighbourhoods.The approach uses neighbourhood discovery and supervision to avoid grouping noise while learning latent class decision boundaries.
- Conclusion: Figure 6 tracks neighbourhood quality over training rounds on CIFAR10.
- Conclusion: Figure 7 visualizes evolving model-learning attention across training rounds for six ImageNet classes and marks a failure case.
- Conclusion: AND avoids grouping noise while retaining clustering’s ability to infer latent class decision boundaries.
- Conclusion: AND outperforms a wide range of state-of-the-art unsupervised deep learning models on image classification benchmarks.The conclusion also reports component analysis aimed at explaining the formulation’s advantages.