Source-linked AI summary
A Unified Objective for Novel Class Discovery
Enrico Fini, Enver Sangineto, Stéphane Lathuilière, Zhun Zhong, Moin Nabi, Elisa Ricci
TL;DR
Novel Class Discovery seeks to classify known labeled classes while discovering disjoint novel classes in unlabeled data, but existing methods commonly use multiple objectives and auxiliary losses. UNO uses multi-view self-labeling so pseudo-labels and ground-truth labels enter one classification objective over all classes. It significantly outperforms related methods across benchmarks, including reported gains of ≈+10% on CIFAR-100 and 8% accuracy on ImageNet.
Problem
Novel Class Discovery must classify labeled classes while discovering disjoint novel classes in unlabeled data, whereas existing approaches commonly use separate objectives for labeled and unlabeled samples.
Method
UNO uses multi-view self-labeling to treat pseudo-labels like ground-truth labels in a unified cross-entropy objective over labeled and unlabeled classes.
Results
UNO surpasses previous methods across three public benchmarks, with reported gains of ≈+10% on CIFAR-100 and 8% accuracy on ImageNet.
Takeaways & Limitations
UNO makes NCD more practical by combining supervised and unsupervised learning in one objective and removing costly self-supervised pretraining.
Takeaways & Limitations
Performance drops considerably on a harder proposed split, indicating that current NCD solutions are not yet ready for deployment.
Abstract
from arXiv · showhide
In this paper, we study the problem of Novel Class Discovery (NCD). NCD aims at inferring novel object categories in an unlabeled set by leveraging from prior knowledge of a labeled set containing different, but related classes. Existing approaches tackle this problem by considering multiple objective functions, usually involving specialized loss terms for the labeled and the unlabeled samples respectively, and often requiring auxiliary regularization terms. In this paper, we depart from this traditional scheme and introduce a UNified Objective function (UNO) for discovering novel classes, with the explicit purpose of favoring synergy between supervised and unsupervised learning. Using a multi-view self-labeling strategy, we generate pseudo-labels that can be treated homogeneously with ground truth labels. This leads to a single classification objective operating on both known and unknown classes. Despite its simplicity, UNO outperforms the state of the art by a significant margin on several benchmarks (~+10% on CIFAR-100 and +8% on ImageNet). The project page is available at: https://ncd-uno.github.io.
1. Introduction
Novel Class Discovery uses labeled classes to help classify known categories and discover disjoint novel classes in unlabeled images. UNO replaces the usual collection of separate objectives with one classification objective over both class sets.
- Novel Class Discovery trains a network to classify labeled classes while discovering new classes in an unlabeled image set.
- Existing NCD methods typically combine supervised learning on labeled data with clustering and auxiliary losses on unlabeled data.These objectives commonly rely on pseudo-labels estimated from the unlabeled set.
- Strong semantic similarity between labeled and unlabeled classes is generally required, while auxiliary self-supervised objectives increase tuning complexity.Some approaches also require unlabeled data during pretraining, limiting sequential use when the unlabeled set changes.
- UNO uses multi-view self-labeling to generate pseudo-labels that are treated like ground-truth labels in a unified cross-entropy loss.Two transformed views are independently clustered, and each view’s assignment provides the other view’s pseudo-label.
- UNO jointly recognizes labeled and unlabeled classes through a single model operating on the complete class set.The paper identifies this joint recognition as a key point often neglected by existing NCD solutions.
- 8% accuracy improvement on ImageNet and ≈+10% on CIFAR-100 are reported over previous methods.The method is also reported to surpass prior work across three public benchmarks and perform even more strongly under changed labeled-to-unlabeled proportions.
2. Related Work
Prior NCD work commonly transfers representations from labeled data and discovers unlabeled classes through pairwise relations, consistency, and multiple losses. UNO instead enforces transformation consistency through pseudo-labeling within a single objective.
- Earlier NCD methods transfer representations from labeled data before learning cluster assignments on unlabeled data.Some approaches use metric learning, fine-tuning, or pairwise similarities to support this transfer.
- Related methods use pseudo-labeling, consistency losses, virtual mixed samples, or contrastive learning to improve novel-class discovery.These techniques address noisy unlabeled labels, transformation invariance, or the availability of positive pairs.
- Existing methods commonly learn novel-class classifiers from unlabeled pairwise relations, enforce transformation consistency, and jointly optimize several losses.
- UNO differs by enforcing data-transformation consistency through pseudo-labeling while using a single objective.
3. Method
UNO formulates NCD as one classification objective over labeled and unlabeled classes. It combines shared representations, multi-view self-labeling, balanced pseudo-labels, and multiple clustering heads.
- Problem formulation: NCD trains a mapping over disjoint labeled and unlabeled classes, with the number of unlabeled classes known a priori.The complete label set contains Cl labeled classes followed by Cu unlabeled classes.
- Architecture: The architecture uses a shared encoder with a labeled classifier head and an unlabeled clustering head.The labeled head has Cl outputs, while the unlabeled head projects features before predicting Cu clusters.
- Unified objective: UNO concatenates labeled and unlabeled logits, applies one temperature-scaled softmax over all classes, and trains with standard cross-entropy.This produces posterior probabilities over the complete label set.
- Unified objective: For labeled images, ground-truth labels are zero-padded; for unlabeled images, zero-padded pseudo-labels provide the corresponding training targets.The padding separates the disjoint known and unknown class groups.
- Multi-view self-labeling: Two augmented views of each image receive the same labeled target or cross-view pseudo-labels, and swapped prediction encourages consistent outputs.The pseudo-label is stop-gradient during loss evaluation, and both view-specific terms instantiate the same objective.
- Pseudo-labeling: Entropy-regularized pseudo-label assignment prevents degenerate predictions by scattering assignments across clusters.Sinkhorn-Knopp solves the assignment problem, and soft pseudo-labels perform best according to the passage.
- Clustering extensions: Overclustering and multiple clustering heads are added to improve representations and reduce sensitivity to suboptimal clustering configurations.The overclustering head uses K = Cu × m outputs, while multiple heads increase the signal reaching shared network components.
4. Experiments
Experiments evaluate UNO on three NCD benchmarks under task-aware and task-agnostic protocols, including ablations of its core components. UNO consistently improves novel-class discovery and representation quality, while unified logit training, overclustering, and strong augmentation contribute to performance.
- Experimental Setup: UNO is evaluated on CIFAR10, CIFAR100, and ImageNet, each split into labeled known classes and unlabeled novel classes.The unlabeled set provides no supervision beyond the number of classes.
- Experimental Setup: The experiments report labeled accuracy and average clustering accuracy for unlabeled classes under task-aware and task-agnostic evaluation.Task-aware evaluation excludes outputs irrelevant to the sample’s known or novel class subset.
- Ablation Study: Separated supervised and clustering objectives cause greater interference than UNO’s concatenated-logit objective, especially under task-agnostic evaluation.The drop is particularly pronounced on the labeled set because separate training does not encourage distinguishing labeled from unlabeled samples.
- Ablation Study: Overclustering improves unlabeled-set performance and overall accuracy, while labeled-class performance does not benefit from fine-grained cluster extraction.Overclustering heads are discarded at test time, while the ordinary clustering heads are retained for evaluation.
- Ablation Study: Strong augmentations consistently increase UNO’s accuracy on both labeled and unlabeled sets, unlike the corresponding augmentation changes for RS.The strong policy adds jittering and greyscale to cropping and flipping, with Gaussian blur also used on ImageNet.
- Comparison with the State of the Art: UNO considerably outperforms the state of the art despite omitting self-supervised pretraining, reaching over 90.0% accuracy on ImageNet.On CIFAR10, its clustering error is reduced to roughly half and approaches supervised accuracy.
- Comparison with the State of the Art: As the number of unlabeled classes increases, task-aware clustering accuracy decreases for all methods, yet UNO remains ahead of the alternatives.The increasing difficulty reflects both harder clustering and fewer labeled classes available for supervised representation learning.
- Qualitative Results: On CIFAR10, UNO produces tighter same-class feature groups and better class separation than RS+ in both shared-feature and logits spaces.RS+ shows entanglement among classes such as cat, dog, and horse.
5. Conclusions
UNO discovers novel classes by combining pseudo-labels with ground-truth labels in one objective, simplifying training while improving cooperation between supervised and unsupervised learning.
- UNO combines pseudo-labels and ground-truth labels in a unified objective for discovering and learning novel classes.
- UNO leverages supervised features from labeled data to discover novel classes in an unlabeled dataset.
- UNO removes the need for costly self-supervised pretraining, making novel class discovery more practical.
- UNO significantly outperforms related methods despite being conceptually simpler and easier to implement and train.
A. Comparison with the state-of-the-art without self-supervised pretraining
Without self-supervised pretraining, UNO is compared with competitors under roughly equal compute and retains a substantial performance advantage across the reported benchmarks.
- 6.7% and 17.6% are UNO’s reported advantages over competitors on CIFAR10 and CIFAR100-20, respectively, with supervised pretraining only.
- Without self-supervised pretraining, all methods except UNO are negatively affected, including RS by ≈6% and DTC by ≈10% on CIFAR100-20.
- The supervised-only comparison trains all methods using roughly the same amount of compute.
- UNO (avg) achieves 96.1±0.5 on CIFAR10, 84.5±1.0 on CIFAR100-20, and 89.2 on ImageNet.
- UNO (best) achieves 96.1±0.5 on CIFAR10, 85.0±0.6 on CIFAR100-20, and 90.6 on ImageNet.
- The benchmark reports task-aware clustering accuracy on CIFAR10, CIFAR100-20, and ImageNet after supervised encoder initialization.
B. Multi-view aggregation strategies
The paper investigates averaging pseudo-labels or logits across views as alternatives to swapped prediction, balancing noise reduction and computational efficiency against training speed and performance.
- Multi-view aggregation strategies generate a single pseudo-label from information across multiple views.
- Averaging pseudo-labels: Averaging pseudo-labels makes the aggregate less noisy because it depends on both views.
- Averaging pseudo-labels: Averaging pseudo-labels also produces more entropic distributions early in training, which slows optimization.
- Averaging logits: Averaging logits requires Sinkhorn-Knopp only once for two views, improving computational efficiency.
- Averaging logits: The logit-averaging strategy does not perform as well as the swapped prediction task.
C. More qualitative results for UNO
UNO’s feature-space analysis shows unlabeled samples organized into separable subgroups, with clustering and overclustering heads shaping their separation differently.
- The analysis concatenates logits from multiple clustering and overclustering heads to visualize the induced feature space.
- Unlabeled classes form subgroups in the shared feature space, with tight clusters that can be separated from other classes.
- Figure A visualizes unlabeled CIFAR10 training samples using t-SNE.
- Clustering head logits: The nonlinear clustering projection head correctly groups most unlabeled samples.
- Overclustering head logits: Overclustering heads increase subgroup separation and sometimes stretch subgroups to minimize the loss.
D. Unknown number of clusters
UNO is evaluated when the number of novel classes is unavailable by estimating the cluster count from a held-out labeled probe subset. Using this estimate, the method is rerun against competing approaches on CIFAR100-20.
- The practical setting is motivated by the fact that the number of unlabeled classes is unlikely to be known in advance.
- The cluster-count estimation approach holds out a probe subset from the labeled data before running constrained semi-supervised clustering.
- k = 23 classes is estimated for CIFAR100-20 using 60 pretraining classes, 20 probe classes, and 20 unlabeled classes.
- The estimated cluster count is then used to rerun UNO and competing methods, with results reported in Tab C.The evaluation concerns CIFAR100-20 under an estimated rather than known class count.
E. Comparison with concurrent works
UNO is compared with OpenMix, NCL, and Jia et al. across CIFAR10, CIFAR100-20, and ImageNet using task-aware evaluation. Despite being simpler, UNO matches or exceeds the concurrent methods overall, with some dataset-specific exceptions.
- UNO is compared against OpenMix, NCL, and Jia et al. on CIFAR10, CIFAR100-20, and ImageNet.
- Despite its simpler design, UNO achieves better or comparable performance than the concurrent methods.
- UNO retains state-of-the-art results on CIFAR10 but is slightly outperformed by NCL on CIFAR100-20 and ImageNet.