Source-linked AI summary
Learning Open Set Network with Discriminative Reciprocal Points
Guangyao Chen, Limeng Qiao, Yemin Shi, Peixi Peng, Jia Li, Tiejun Huang, Shiliang Pu, Yonghong Tian
TL;DR
Open set recognition requires recognizing known classes while rejecting unseen samples, but modeling unknown space from limited known data remains difficult. The paper introduces Reciprocal Point Learning, which represents extra-class space with reciprocal points and bounds it through multi-category interaction, plus the Air-300 dataset. Across multiple datasets, the method outperforms previous state-of-the-art open set classifiers.
Problem
Open set recognition must classify predefined classes and reject unseen classes, while open space risk remains difficult to model using only known training data.
Method
Reciprocal Point Learning represents each known category’s extra-class space with a reciprocal point and uses a bounded-space regularization framework with multi-category interaction.
Results
The method outperforms previous state-of-the-art open set classifiers across multiple datasets; on Air-300, its AUROC is about 6% higher than SoftMax.
Takeaways & Limitations
Reciprocal points introduce unknown information using only known classes, supporting more compact feature spaces that separate known and unknown samples.
Takeaways & Limitations
The framework treats samples outside the labeled known-data set as potential unknown data.
Abstract
from arXiv · showhide
Open set recognition is an emerging research area that aims to simultaneously classify samples from predefined classes and identify the rest as 'unknown'. In this process, one of the key challenges is to reduce the risk of generalizing the inherent characteristics of numerous unknown samples learned from a small amount of known data. In this paper, we propose a new concept, Reciprocal Point, which is the potential representation of the extra-class space corresponding to each known category. The sample can be classified to known or unknown by the otherness with reciprocal points. To tackle the open set problem, we offer a novel open space risk regularization term. Based on the bounded space constructed by reciprocal points, the risk of unknown is reduced through multi-category interaction. The novel learning framework called Reciprocal Point Learning (RPL), which can indirectly introduce the unknown information into the learner with only known classes, so as to learn more compact and discriminative representations. Moreover, we further construct a new large-scale challenging aircraft dataset for open set recognition: Aircraft 300 (Air-300). Extensive experiments on multiple benchmark datasets indicate that our framework is significantly superior to other existing approaches and achieves state-of-the-art performance on standard open set benchmarks.
1 Introduction
The paper frames open set recognition as classifying known samples while rejecting unseen classes, then proposes Reciprocal Point Learning to model extra-class space using only known classes. It also introduces Air-300 and reports superior performance across benchmark datasets.
- Motivation: Open set recognition must classify known classes and reject unseen or unknown samples during testing.
- Motivation: SoftMax is closed over known classes, while generative approaches for unknown samples can be unstable; modeling open space risk from only known data remains difficult.
- Reciprocal Point Learning: Reciprocal Points represent each known category’s extra-class space, identifying samples through their otherness to these points.
- Reciprocal Point Learning: RPL bounds each known class’s extra-class embedding space with reciprocal points and a learnable margin, using multi-class interaction to constrain unknown space.
- Dataset: Air-300 contains 320,000 annotated color images from 300 aircraft classes, with long-tailed category sizes designed to simulate the real visual world.
- Results: Experiments on multiple benchmark datasets show that RPL significantly outperforms existing state-of-the-art deep open set classifiers.
2 Related Work
Related work covers open set recognition, out-of-distribution detection, and prototype learning as neighboring approaches to recognizing known and unfamiliar samples.
- Open Set Recognition: Open set recognition formalizes classification with a rejection option for samples from classes absent during training.
- Out-of-Distribution Detection: OOD detection addresses test samples that do not belong to the training set, often using anomaly scores from deep classifiers.
- Prototype Learning: Prototype learning uses learnable latent vectors or class centers to reduce intra-class variance and improve feature discrimination.
3 Reciprocal Point Learning
Reciprocal Point Learning represents each known class's extra-class space with learnable reciprocal points and classifies samples by their distance-based otherness. Its joint objective combines known-data classification with open-space-risk reduction using bounded spaces around reciprocal points.
- Open Set Learning: Open set recognition minimizes empirical risk on labeled known data together with open-space risk on potential unknown data.The formulation aims to make known and unknown spaces more distinguishable.
- Reciprocal Points: Reciprocal points represent latent extra-class space for each known category, corresponding to samples outside that category's embedding space.For class k, Pk contains M learnable reciprocal points associated with the complementary space.
- Reciprocal Points for Classification: Samples are assigned to known categories according to their distance-based otherness from each class's reciprocal points.The framework estimates otherness across all known classes; greater distance is associated with a greater probability of assignment to that class.
- Reciprocal Points for Classification: Reciprocal-point classification maximizes separation between known samples and reciprocal points, widening the interval between closed and open spaces.The loss also corresponds to the empirical classification-risk term in the overall objective.
- Reducing Open Space Risk: Open-space risk is indirectly bounded by constraining distances between known samples and reciprocal points within learnable margins.Directly restricting the open space is impractical because it contains many potentially unknown samples.
- Optimization: The overall RPL loss combines reciprocal-point classification loss with open-space regularization, while training updates convolutional, reciprocal-point, and margin parameters.The algorithm jointly optimizes the model parameters, reciprocal points, and learnable margins.
4 Experiments
Experiments evaluate RPL for open set identification and open long-tailed recognition using benchmark datasets and the Air-300 dataset. Results show strong performance, while analyses examine bounded feature spaces, λ, reciprocal-point counts, and prototype assistance.
- Evaluation protocol: AUROC is the evaluation metric for open set identification, averaged over five randomized trials.AUROC is threshold-independent and measures whether positive examples receive higher detection scores than negative examples.
- Open set identification: RPL outperforms competing methods across all datasets, including generation-based and encoder-decoder approaches, while using only known training samples and a single encoder.RPL-WRN also performs significantly better than recent state-of-the-art methods on SVHN, CIFAR, and TinyImageNet.
- Open long-tailed recognition: Air-300 contains 320,000 annotated images across 300 aircraft classes with a long-tailed distribution, split into 180 known training classes and 120 novel unknown testing classes.The dataset also provides fine-grained and coarse-grained labels for aircraft models and superclasses.
- Open long-tailed recognition: RPL is about 6% higher than Softmax in AUROC on Air-300 and improves classification and open set identification by nearly 2% on ImageNet-LT.The reported results indicate effectiveness on long-tailed open set recognition despite ImageNet-LT's many categories and fewer training samples.
- Reducing open space risk: RPL with open-space regularization confines unknown and known features to limited, separated ranges, preventing unknown samples from receiving high-confidence known-class predictions.Without the regularizer, reciprocal points collapse toward the origin and known and unknown features overlap substantially.
- Further analysis: Increasing λ improves open set identification but can slightly reduce known-class accuracy when too large, while more known classes require a larger margin.Open set identification remains relatively stable as the number of reciprocal points increases; RPL++ further improves RPL by adding prototypes.
5 Conclusion
The paper introduces Reciprocal Points and Reciprocal Point Learning to separate known and unknown classes, reports superior performance, and contributes the Air-300 dataset.
- Reciprocal Point represents the extra-class space corresponding to each known category.
- RPL introduces unknown information through reciprocal points to optimize a feature space separating known and unknown samples.
- Experiments on multiple datasets show that RPL outperforms previous state-of-the-art open set classifiers in all cases.
- Air-300 is an open long-tailed dataset designed to simulate natural data distributions for open set recognition and other visual tasks.
A Preliminaries
The framework formulates open set recognition through category-wise binary risks and combines classification, open-space, and prototype objectives to distinguish known from unknown spaces.
- Open set recognition integrates multiple one-versus-rest binary classification tasks into a multiclass recognition problem.
- The formulation separates known space, open positive space, and infinite open negative space for each category.
- Minimizing empirical classification risk together with open space risk makes known and unknown spaces more distinguishable.
- Reciprocal points and prototypes are complementary representations for extra-class and known-class structure.
- Prototype Loss reduces intra-class variance after reciprocal points establish the embedding structure.The combined objective is L(x; θ, P, R, M) = Lc(x; θ, P) + λLo(x; θ, P, R) + βLpl(x; θ, M).
C Implementation Details
Implementation uses specified reciprocal-point settings and visualizes RPL representations across regularization strengths, known-class counts, and training stages.
- Reciprocal points use γ = 0.5, random-normal initialization, zero-initialized margins, and λ = 0.1 for out-of-distribution experiments.Features use the global-average-pooling output of WRN-40-4.
- Figure 6 varies λ with six known and four unknown MNIST classes, then varies known-class counts while keeping λ = 0.1.Colored triangles represent reciprocal points for different known classes.
- Figure 7 compares learned Air-300 embeddings across training stages with 180 known training classes and 120 novel unknown testing classes.Colored triangles represent reciprocal points for known classes.
D.1 Ablation Study
RPL learns bounded embedding spaces in which unknown samples occupy an internal region while known classes form separated peripheral distributions; these structures change with class count and training.
- RPL confines unknown samples to an internal bounded space and distributes known classes around its periphery.The bounded space size is controlled by Lo.
- With more known classes, known-class distributions widen and the learned margin R grows.
- On Air-300, RPL makes known and unknown features more clearly separated than the network’s initial features.
- Reciprocal points lie almost within the embedding space of unknown classes, indicating that RPL introduces unknown information during training.
D.2 The Long-tailed Recognition
RPL is evaluated under long-tailed distributions designed to simulate practical real-world conditions. The results indicate that it distinguishes unknown categories while maintaining similar or better classification performance.
- Long-tail experiments simulate real-world distributions to evaluate RPL’s practical effectiveness.
- RPL distinguishes unknown categories in long-tail scenarios.
- On Air-300, RPL recognizes unknowns while maintaining similar or even better classification performance across head and tail categories.The experiments separately evaluate the head 80% and tail 20% categories using the same setting as Section 4.2.
D.3 Open Set Recognition
The study evaluates F1-score trends across varying openness on CIFAR100 using repeated randomized trials and a uniform threshold. RPL and RPL++ outperform Softmax and GCPL as openness increases.
- Table 4 reports F1-score trends under varying openness on CIFAR100, averaged over ten randomized trials.
- Each trial selects 15 known categories and samples unknowns from the remaining categories according to openness.
- All methods use ResNet18 backbones, a 0.1 threshold, and softmax-normalized outputs for uniform threshold evaluation.
- RPL and RPL++ are superior to Softmax and GCPL as openness increases.