PaperlayerView exact revision on arXiv

Source-linked AI summary

Parametric Classification for Generalized Category Discovery: A Baseline Study

Xin Wen, Bingchen Zhao, Xiaojuan Qi

arXiv:2211.11727v4cs.CVcs.LG

TL;DR

GCD must classify seen categories while discovering novel ones, but prior work attributed parametric failure to overfitting and favored non-parametric classification. This paper traces the failure to biased unreliable pseudo-labels, then introduces a simple entropy-regularised parametric method that achieves state-of-the-art performance across GCD benchmarks and remains robust to unknown category counts.

  • Problem

    GCD requires joint classification of seen categories and discovery of novel categories, while prior parametric classifiers were reported to overfit seen classes and non-parametric alternatives impose computational costs.

  • Method

    The method retains GCD representation learning, uses cross-entropy and self-distillation classification objectives, and adds entropy regularisation to counter biased predictions.

  • Results

    The method achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown category numbers.

  • Takeaways & Limitations

    Unreliable pseudo-labels with Old-class and class-distribution biases are identified as the crucial factor behind previous parametric failures, supporting parametric classification as a strong GCD baseline.

  • Takeaways & Limitations

    Representation learning follows prior GCD work, leaving potential improvements from stronger augmentations, larger batches, and better alignment with target classification criteria.

Abstract

from arXiv · show

Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. Previous studies argued that parametric classifiers are prone to overfitting to seen categories, and endorsed using a non-parametric classifier formed with semi-supervised k-means. However, in this study, we investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We demonstrate that two prediction biases exist: the classifier tends to predict seen classes more often, and produces an imbalanced distribution across seen and novel categories. Based on these findings, we propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers. We hope the investigation and proposed simple framework can serve as a strong baseline to facilitate future studies in this field. Our code is available at: https://github.com/CVMI-Lab/SimGCD.

1. Introduction

GCD extends semi-supervised recognition by requiring models to classify seen categories while discovering novel categories in unlabelled data. This study revisits parametric classification, identifies biased and unreliable pseudo-labels as the key failure, and proposes a simple entropy-regularised baseline.

  • Motivation: GCD uses unlabelled data that may contain categories distinct from labelled categories, aiming to classify seen classes and discover novel ones jointly.The setting relaxes the standard semi-supervised assumption that labelled examples exist for every category.
  • Prior approaches: Parametric classifiers can overfit to seen categories, whereas semi-supervised k-means reduces this issue but incurs quadratic clustering cost and loses learnable joint optimisation.The paper revisits whether non-parametric classification is necessary for GCD.
  • Investigation: High-quality supervision validates prior representation and training choices, while unreliable pseudo-labels explain degraded parametric-classifier performance on novel classes.The diagnosed biases are excessive predictions of Old classes and imbalanced pseudo-labels across classes.
  • Method: The proposed parametric baseline combines GCD representation learning, cross-entropy for labelled samples, self-distillation for unlabelled samples, and entropy regularisation.Entropy regularisation encourages more uniformly distributed predictions across categories.
  • Results: The method achieves state-of-the-art performance on multiple GCD benchmarks and shows that parametric classification can work well on both Old and New classes.The performance overview contrasts this result with UNO+ and non-parametric GCD.

2. Related Works

Related work spans semi-supervised learning, open-set semi-supervised learning, GCD, novel category discovery, and deep clustering. GCD differs by requiring discovery and classification of novel categories rather than merely rejecting outliers or assuming disjoint class sets.

  • Semi-Supervised Learning: Semi-supervised learning assumes labelled instances are available for every category represented in the unlabelled dataset.Consistency-based methods train on multiple augmentations and can benefit from self-supervised representations.
  • Open-Set Semi-Supervised Learning: Open-set semi-supervised learning handles unlabelled outliers by learning the labelled categories, while novel-category outliers are rejected rather than classified.Its objective therefore differs from GCD’s discovery requirement.
  • Generalized Category Discovery: GCD allows unlabelled data to contain similar yet distinct categories from labelled data, making it more challenging than common semi-supervised learning.It extends novel category discovery, where labelled and unlabelled class sets do not overlap.
  • Generalized Category Discovery: Prior GCD work reported that semi-supervised k-means could outperform parametric baselines by alleviating overfitting to seen categories.This paper revisits that conclusion while considering computational cost and learnability.
  • Deep Clustering: Deep clustering emphasizes reliable pseudo-labels, progressing from k-means hard labels toward optimal-transport soft labels and teacher-model predictions.These approaches also support unsupervised representation learning.

3. On the Failure of Parametric Classification

The investigation shows that representation and joint-training choices can work under strong supervision, but unreliable pseudo-labels drive parametric failure. Two prediction biases—toward Old classes and toward imbalanced class frequencies—motivate the proposed diagnosis.

  • 3.1. Investigation Setting: GCD assumes labelled and unlabelled datasets have nested label spaces, with the unlabelled category count commonly known or estimated beforehand.The investigation varies supervision quality from labelled-only to oracle and pseudo-labelled settings.
  • 3.1. Investigation Setting: The classifier uses a prototypical cosine-logit formulation with temperature scaling, while representation learning follows GCD’s supervised and self-supervised contrastive objectives.The temperature scales logits to facilitate cross-entropy optimisation.
  • 3.2. Which Representation to Build Your Classifier?: Post-backbone representations improve Old-class results consistently, but New-class benefits require stronger pseudo-labels.The post-projector versus post-backbone comparison is evaluated under varying supervision quality.
  • 3.4. The Devil Is in the Biased Predictions: Parametric methods exhibit two biases: frequent False Old predictions and highly imbalanced predictions across classes, including notable within-New misclassification.These findings motivate prediction-entropy regularisation.

4. Method

The method is a one-stage GCD framework that jointly learns representations and a parametric classifier using self-distillation and entropy regularisation. It uses contrastive learning for representations, sharpened cross-view predictions for unlabelled pseudo-labels, ground-truth labels for labelled samples, and entropy regularisation to address prediction bias.

  • Overall framework: The framework jointly trains a parametric classifier with self-distillation and entropy regularisation in a single stage.It builds on GCD and combines representation learning with classification objectives.
  • Representation learning: Representation learning combines supervised contrastive learning on labelled samples with self-supervised contrastive learning on all samples.The supervised objective matches positives by label, while the self-supervised objective matches two augmented views of the same image.
  • Representation learning: The self-supervised contrastive loss compares two augmented views of each image using ℓ2-normalised projected features and a temperature parameter.The projected feature is z_i = g(f(x_i)), where f is the backbone and g is the projection head.
  • Parametric classification: For classification, labelled samples use ground-truth labels, whereas unlabelled samples use sharpened predictions from another augmented view as soft pseudo-labels.Predictions are computed from cosine similarities between normalised hidden features and randomly initialised category prototypes, scaled by τ_s.
  • Parametric classification: The classification objective combines cross-entropy on pseudo-labels or ground-truth labels with mean-entropy maximisation over batch predictions.The overall objective combines representation learning and classification losses, with λ balancing supervised and unsupervised terms.
  • Discussion: The authors relate the framework to consistency-based learning but distinguish it by jointly performing category discovery and self-training while using entropy regularisation to overcome prediction bias.They report that entropy regularisation helps address bias between and within seen and novel classes.

5. Experiments

Experiments evaluate SimGCD across diverse GCD benchmarks, compare it with established methods, and analyze efficiency, component contributions, prediction bias, category-number robustness, representation changes, and Old–New trade-offs.

  • Experimental Setup: The evaluation spans generic recognition, semantic-shift, long-tailed fine-grained, and large-scale datasets, using labelled Old classes and unlabelled data containing remaining instances.The protocol samples labelled classes, uses half their images for labelled training, and treats the remaining images as unlabelled.
  • Comparison With the State of the Arts: SimGCD improves New-class recognition by around 10% over state-of-the-art methods on both semantic-shift and generic recognition benchmarks, while remaining competitive on Old classes.The compared methods include ORCA, GCD, RS+, UNO+, and k-means on DINO features.
  • Comparison With the State of the Arts: SimGCD shows consistent improvements across all metrics on Herbarium 19 and ImageNet-1K, extending gains to long-tailed and large-scale settings.Herbarium 19 is described as closer to real-world GCD applications, while ImageNet-1K provides a large-scale generic classification setting.
  • Efficiency: SimGCD reduces inference complexity from GCD’s O(N^2d + NKdt) to O(N_uKd) by replacing k-means with nearest-neighbour prototype search.The comparison concerns inference over the unlabelled split and assumes GPU implementations for all methods.
  • Ablation Study: Replacing self-labelling with self-distillation improves New-class accuracy by 26% on CIFAR100 and 13% on CUB, while teacher warmup mainly helps fine-grained datasets.The warmup lowers pseudo-label confidence early, when the classifier and representation are not yet well fitted to the target data.
  • Ablation Study: Jointly training representations with classification consistently improves New classes, although CIFAR100 shows a notable Old-class drop attributed to incorrect pseudo-labels.The joint objective produces less ambiguous representations with larger margins and more compact clusters in the broader analysis.
  • Analyses And Discussions: Entropy regularisation reduces False New and False Old errors, produces class distributions closer to ground truth, and improves robustness when the category number is unknown.With stronger entropy regularisation, the model remains robust to category-number misspecification and keeps active prototypes near the true category number.
  • Analyses And Discussions: With identical representations, GCD produces highly imbalanced predictions, whereas SimGCD better matches the true distribution and improves GCD’s tail classes.The analysis identifies prediction distribution as a source of the performance gap between the non-parametric and parametric classifiers.

6. Limitations and Potential Future Works

The paper identifies several boundaries for future work, including representation quality, alignment between discovered and human-defined categories, and fairness in low-data or long-tailed settings.

  • Representation learning: Representation learning remains a limitation because the paper follows prior GCD choices, including a batch size of 128 that might constrain learned representations.The authors suggest stronger augmentations, multiple local crops, and larger batches as possible directions.
  • Representation learning: Advances in representation learning could lead to further gains beyond the supervised-plus-self-supervised contrastive paradigm used here.
  • Alignment to human-defined categories: Seen-category labels may poorly guide novel-category discovery when the seen and novel categories differ substantially, such as ImageNet-to-CUB transfer.
  • Alignment to human-defined categories: Matching discovered categories to human-defined names remains necessary, while exploiting relationships among class names and text-space prediction are proposed as future directions.
  • Ethical considerations: Low-data or long-tailed settings may permit discrimination against minority categories or instances, depending on the task data and classification criteria.

7. Conclusion

The study attributes previous parametric classifiers’ failures in GCD to biased, unreliable pseudo-labels and proposes a simple parametric baseline that performs strongly across multiple benchmarks.

  • The study identifies unreliable pseudo-labels with significant biases as the crucial factor behind previous parametric classifiers’ degraded novel-class recognition.
  • The proposed simple parametric classification method achieves state-of-the-art performance on multiple GCD benchmarks.
  • The framework is intended to provide a strong baseline for future research on robust and accurate category discovery.

Supplementary Material

The supplementary material includes an author listing for the paper.

  • The supplementary material lists Xin Wen, Bingchen Zhao, and Xiaojuan Qi as authors.

A.1. Experiment Setting Details

The experiments divide categories into labelled Old and unlabelled New groups, with dataset-specific sampling for CIFAR100, and adapt baseline evaluation procedures to available hardware.

  • 50% of classes are sampled as labelled Old categories and the remainder as unlabelled New categories under the standard GCD split.
  • CIFAR100 instead uses an 80% Old-category split following novel category discovery practice.
  • For generic object-recognition datasets, labelled classes are selected by class index, specifically the first |Yl| classes.
  • ImageNet-1K baseline evaluation omits k-means++ initialization because its GPU memory requirements exceed available hardware, using a faiss reimplementation instead.

A.3. Error Analysis Details

The error analysis constructs a normalized confusion matrix to separate prediction errors between and within seen (“Old”) and novel (“New”) classes. This decomposition distinguishes different sources of prediction bias in the model.

  • Prediction errors are obtained by normalizing a full confusion matrix and aggregating its four Old/New submatrices.Classes are ordered with Old classes first and New classes afterward; each matrix element is divided by the number of testing samples.
  • The resulting error matrix separates misclassification between Old and New categories from errors within each category group.The decomposition is designed to distinguish prediction bias between and within seen and novel categories.
  • The analysis treats diagonal elements such as “True Old” as error categories rather than correct predictions.

B.1. Main Results

The supplementary results report SimGCD across independent runs, unknown-category-number settings, and multiple benchmark datasets. The method shows consistent improvements when the category number is estimated either with a specialised algorithm or with a loose two-times-ground-truth estimate.

  • SimGCD results are reported from three independent runs to avoid randomness.
  • The improvements hold with either a specialised category-number estimate or a loose estimate equal to two times the ground-truth K.Other category-number estimates are also reported as applicable because the method is robust to a wide range of estimations.
  • SimGCD improves consistently on CIFAR100, ImageNet-100, CUB, and Stanford Cars when the unknown category number is estimated.The comparison covers both specialised estimation and loose category-number settings.
  • This robustness to unknown category numbers could ease deployment of parametric GCD classifiers in real-world scenarios.

B.3. Extended Analyses

Extended analyses examine entropy regularisation, prediction distributions, and category-number estimates across coarse-grained, fine-grained, and long-tailed datasets. The results show broad bias reduction but also dataset-dependent effects, especially under long-tailed or overly strong regularisation settings.

  • Entropy regularisation mainly reduces Old/New misclassification and errors within New categories across five representative datasets.The analysis covers coarse-grained, fine-grained, and long-tailed classification tasks.
  • On long-tailed Herbarium 19, False Old errors increase, although gains in distinguishing novel categories remain consistent.The authors associate the increased difficulty with the dataset’s long-tailed distribution.
  • Entropy regularisation alleviates prediction bias within both Old and New classes and better fits the ground-truth class distribution.
  • Too much entropy regularisation can increase prediction bias on class-balanced ImageNet-100, while helping fit the distribution of long-tailed Herbarium 19.The regulariser is formulated to move predictions toward a uniform distribution, but its empirical effect depends on the dataset.
  • Using a category number greater than ground truth may harm distribution fitting on ImageNet-100 but help on Herbarium 19, while the model still identifies the ground-truth number on ImageNet-100.
  • GCD cannot directly use prior class distributions because novel-class labels are unknown, and online estimates are inaccurate in the open-world setting.
Loading 2211.11727v4…