Source-linked AI summary
OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in An Open World
Zhun Zhong, Linchao Zhu, Zhiming Luo, Shaozi Li, Yi Yang, Nicu Sebe
TL;DR
Novel class discovery must identify classes in unlabeled data using labeled data from disjoint classes, but existing methods largely underuse those labels during clustering. OpenMix mixes known- and novel-class examples and their labels, then uses reliable anchors for further combinations; experiments on three benchmarks show consistent performance improvements and superiority to state-of-the-art methods.
Problem
Novel class discovery asks how to identify new classes in unlabeled data using labeled data from disjoint known classes, when existing methods largely ignore labeled data during unsupervised clustering.
Method
OpenMix dynamically mixes labeled known-class samples with unlabeled novel-class samples using joint label distributions, then mixes reliable unlabeled anchors with other unlabeled samples.
Results
Across three image-classification benchmarks, OpenMix consistently improves performance and outperforms state-of-the-art methods in novel class discovery.
Takeaways & Limitations
OpenMix provides a way to reuse known-class knowledge while generating more reliable pseudo-labels and diverse smooth samples for novel-class recognition.
Takeaways & Limitations
MixUp does not guarantee lower label error for every generated sample, so the mixed sample's label error can exceed that of its unlabeled counterpart.
Abstract
from arXiv · showhide
In this paper, we tackle the problem of discovering new classes in unlabeled visual data given labeled data from disjoint classes. Existing methods typically first pre-train a model with labeled data, and then identify new classes in unlabeled data via unsupervised clustering. However, the labeled data that provide essential knowledge are often underexplored in the second step. The challenge is that the labeled and unlabeled examples are from non-overlapping classes, which makes it difficult to build the learning relationship between them. In this work, we introduce OpenMix to mix the unlabeled examples from an open set and the labeled examples from known classes, where their non-overlapping labels and pseudo-labels are simultaneously mixed into a joint label distribution. OpenMix dynamically compounds examples in two ways. First, we produce mixed training images by incorporating labeled examples with unlabeled examples. With the benefits of unique prior knowledge in novel class discovery, the generated pseudo-labels will be more credible than the original unlabeled predictions. As a result, OpenMix helps to prevent the model from overfitting on unlabeled samples that may be assigned with wrong pseudo-labels. Second, the first way encourages the unlabeled examples with high class-probabilities to have considerable accuracy. We introduce these examples as reliable anchors and further integrate them with unlabeled samples. This enables us to generate more combinations in unlabeled examples and exploit finer object relations among the new classes. Experiments on three classification datasets demonstrate the effectiveness of the proposed OpenMix, which is superior to state-of-the-art methods in novel class discovery.
1 Introduction
Novel class discovery seeks to identify previously undefined classes in unlabeled data using labeled examples from disjoint known classes. OpenMix addresses the underuse of known-class data by mixing labeled and unlabeled samples with joint label distributions and reliable pseudo-labels.
- 1 Introduction: Novel class discovery identifies new classes in unlabeled data with support from labeled data belonging to disjoint known classes.Existing methods commonly pre-train on labeled data and then cluster the unlabeled data.
- 1 Introduction: Directly mixing unlabeled samples can propagate uncertain pseudo-labels, whereas OpenMix produces high-confidence old-class labels and more reliable new-class labels.Mixing with labeled samples partially eliminates uncertainty in the novel-class pseudo-labels and helps prevent overfitting to wrong assignments.
- 1 Introduction: OpenMix mixes labeled known-class samples with unlabeled novel-class samples after extending labels and pseudo-labels into a joint label distribution.The method uses the clean labels of known classes and the distinct label spaces of labeled and unlabeled data.
- 1 Introduction: OpenMix selects high-probability unlabeled examples as reliable anchors and mixes them with other unlabeled samples to generate diverse smooth novel-class samples.This second mixing strategy exploits finer relations among new classes.
- 1 Introduction: Across three datasets, OpenMix outperforms state-of-the-art methods by a large margin in novel class discovery.The contribution statement reports consistent improvement from leveraging known knowledge in the task.
2 Related Work
Novel class discovery differs from related settings because labeled and unlabeled data belong to completely different class sets. OpenMix applies MixUp-style sample construction to use known-class labels during discovery of novel classes.
- 2 Related Work: Novel class discovery supplies labeled data from known classes and unlabeled data from entirely different novel classes, unlike semi-supervised learning.Unlike transfer learning, the target data in novel class discovery are unlabeled.
- 2 Related Work: Existing novel-class-discovery methods generally use labeled data for initialization and then perform unsupervised clustering, with limited use during clustering itself.Prior work mainly uses labeled data to obtain features, pairwise supervision, or preserve old-class accuracy.
- 2 Related Work: Unsupervised clustering lacks the labeled prior knowledge available in novel class discovery and may admit criteria based on color, shape, or other attributes.Novel class discovery is therefore distinguished by its additional known-class information.
- 2 Related Work: Existing MixUp-based methods assume clean labels for every class of interest, so they cannot be directly applied when novel-class labels are unavailable.OpenMix adapts MixUp to leverage known-class labeled data under this disjoint-label setting.
3 Our Method
OpenMix extends a two-stage novel class discovery baseline by reusing labeled old-class knowledge during unsupervised clustering of disjoint unlabeled new classes. It mixes labeled and unlabeled examples with joint label distributions, then uses reliable anchors to create additional new-class combinations.
- 3.1 Baseline: The baseline first initializes a CNN and old classifier on labeled data, then adds a new-class classifier for unsupervised clustering of unlabeled data.The clustering stage uses pseudo-pair learning and pseudo-label learning to identify samples from new classes.
- 3.1 Baseline: Pseudo-pair learning labels image pairs as positive or negative according to whether their cosine similarity exceeds threshold θ1, then trains with binary cross-entropy.This converts pairwise similarity into supervision for the new-class classifier.
- 3.1 Baseline: The baseline’s optimal new-class predictions are Cu-diverse one-hot vectors, automatically partitioning unlabeled data into Cu classes.Pseudo-label learning trains only on unlabeled samples assigned one-hot pseudo-labels and combines with pseudo-pair learning in the unsupervised loss.
- 3.2 OpenMix: OpenMix mixes labeled and unlabeled samples after extending their disjoint labels and pseudo-labels into a joint label distribution.The generated sample and pseudo-label are formed with MixUp, while the mixing constraint keeps the sample closer to the labeled example.
- 3.2 OpenMix: OpenMix produces more reliable pseudo-labels than unlabeled counterparts because labeled samples provide exact old-class information and belong to disjoint classes.This reduces overfitting to wrongly assigned unlabeled pseudo-labels; the overall objective balances the baseline and OpenMix losses with λ2.
- 3.4 Discussion: OpenMix’s label error is always lower than its unlabeled counterpart’s unless that counterpart’s pseudo-label is already exactly accurate.Direct MixUp among unlabeled samples has no such guarantee; its label error can fail to decrease.
4 Experiments
Experiments on CIFAR-10, CIFAR-100, and ImageNet evaluate OpenMix through ablations, sensitivity analysis, visualizations, and comparisons with state-of-the-art methods. OpenMix consistently improves novel class discovery and achieves the strongest reported clustering accuracy across all three datasets.
- Ablation Study: Pseudo-pair learning is essential: removing it reduces CIFAR-10 ACC from 90.9% to 70.8% and produces only 23.9% ACC on CIFAR-100.Removing pseudo-label learning lowers CIFAR-100 performance by 4% ACC and 0.051 NMI, while combining both components performs best.
- Ablation Study: 2.4% and 3.3% baseline ACC improvements on CIFAR-10 and CIFAR-100 result from mixing labeled and unlabeled samples with OpenMix without anchors.Adding anchor-based mixing further improves performance, with the full method exceeding the no-anchor variant by 2% and 2.7%.
- Anchor Investigation: Selected-anchor accuracy stays above 95% for the no-anchor-mixing variant, whereas full OpenMix slightly reduces accuracy after 20 epochs.The no-anchor variant also achieves much higher anchor accuracy than the baseline throughout training.
- Sensitivity Analysis: OpenMix improves performance for every tested weight λ2 ≥ 1 and becomes stable when λ2 ≥ 1000, with best results between 1000 and 3000.The weight is varied from 0 to 5000 while other hyperparameters remain fixed.
- Comparison with State-of-the-Art: 95.3% on CIFAR-10, 90.1% on CIFAR-100, and 85.7% on ImageNet: OpenMix outperforms state-of-the-art methods in clustering accuracy.The method surpasses DTC on all three datasets and exceeds RS by 3.6% on CIFAR-10 and 3.2% on ImageNet.
- Visualization: t-SNE visualizations show unlabeled samples progressively grouping by class as training proceeds, forming more discriminative clusters.The visualization uses CIFAR-10 unlabeled samples, with colors representing classes.
5 Conclusions
OpenMix addresses novel class discovery by incorporating labeled known-class data into unsupervised clustering of unlabeled novel-class data. It generates joint-class samples with reliable pseudo-labels and diverse smooth novel-class samples, improving recognition and achieving state-of-the-art results on three benchmarks.
- OpenMix incorporates labeled known-class and unlabeled novel-class samples, together with their labels and pseudo-labels, during unsupervised clustering.
- The method generates joint-class samples with reliable pseudo-labels and diverse smooth samples of new classes.
- Experiments on three image-classification benchmarks show consistent baseline improvements and state-of-the-art results.