PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Reciprocal Points Learning for Open Set Recognition

Guangyao Chen, Peixi Peng, Xiangqian Wang, Yonghong Tian

arXiv:2103.00953v3cs.CV

TL;DR

OSR must classify seen classes while identifying unseen classes as unknown, requiring simultaneous control of known-data classification risk and unknown-data open space risk. The paper proposes ARPL, which models extra-class space with Reciprocal Points, applies adversarial margin constraints, and generates confusing samples; experiments on benchmark datasets report superiority over existing approaches and state-of-the-art performance.

  • Problem

    OSR must simultaneously classify labeled known data correctly and identify unseen classes as unknown while reducing both empirical classification risk and open space risk.

  • Method

    ARPL models latent extra-class space with Reciprocal Points, uses adversarial margin constraints among known categories, and generates confusing samples through instantiated adversarial enhancement.

  • Results

    The method is reported as significantly superior to existing approaches and achieves state-of-the-art performance across benchmark datasets.

  • Takeaways & Limitations

    Reciprocal Points and adversarial enhancement provide a framework for separating known features from diverse unknown distributions while retaining known-class recognition.

  • Takeaways & Limitations

    Unknown classes very similar to known classes remain challenging because limited known priors can produce high feature responses.

Abstract

from arXiv · show

Open set recognition (OSR), aiming to simultaneously classify the seen classes and identify the unseen classes as 'unknown', is essential for reliable machine learning.The key challenge of OSR is how to reduce the empirical classification risk on the labeled known data and the open space risk on the potential unknown data simultaneously. To handle the challenge, we formulate the open space risk problem from the perspective of multi-class integration, and model the unexploited extra-class space with a novel concept Reciprocal Point. Follow this, a novel learning framework, termed Adversarial Reciprocal Point Learning (ARPL), is proposed to minimize the overlap of known distribution and unknown distributions without loss of known classification accuracy. Specifically, each reciprocal point is learned by the extra-class space with the corresponding known category, and the confrontation among multiple known categories are employed to reduce the empirical classification risk. Then, an adversarial margin constraint is proposed to reduce the open space risk by limiting the latent open space constructed by reciprocal points. To further estimate the unknown distribution from open space, an instantiated adversarial enhancement method is designed to generate diverse and confusing training samples, based on the adversarial mechanism between the reciprocal points and known classes. This can effectively enhance the model distinguishability to the unknown classes. Extensive experimental results on various benchmark datasets indicate that the proposed method is significantly superior to other existing approaches and achieves state-of-the-art performance.

1 INTRODUCTION

Open set recognition must classify known classes while rejecting unseen classes, but methods focused only on known data can leave known and unknown features overlapping. ARPL models latent extra-class space with Reciprocal Points, constrains it adversarially, and generates confusing samples to improve separation.

  • Motivation: OSR must jointly reduce empirical classification risk on labeled known data and open space risk on potential unknown data.Open space risk concerns labeling open space as positive for a known class.
  • Limitations of Existing Methods: Softmax encourages feature separability but does not sufficiently distinguish known and unknown classes, while prototype methods can place prototypes within unknown-class space.Both approaches focus on known data and therefore can leave open space risk insufficiently controlled.
  • Reciprocal Points: Reciprocal Points model latent extra-class space as representative features of non-cats or, more generally, features adverse to known-class prototypes.They identify known classes by otherness and instantiate otherwise unexploited extra-class space.
  • Empirical Illustration: Figure 1 reports that ARPL achieves better separation of known and varied unknown deep features than Softmax or Prototype Learning.The evaluated unknown datasets are KMNIST, SVHN, and CIFAR-100, with decreasing similarity to MNIST.
  • ARPL Framework: ARPL uses confrontation among multiple known classes and Reciprocal Points to enlarge target-to-reciprocal distances and reduce empirical and open space risks.Its adversarial margin constraint limits the latent open space constructed by reciprocal points.
  • Adversarial Enhancement: Instantiated adversarial enhancement generates diverse, confusing training samples from confrontation between known data and Reciprocal Points to improve known–unknown distinguishability.The strategy is introduced as a novel adversarial training mechanism for enhancing model distinguishability.

2 RELATED WORK

Related work places OSR among discriminative and generative approaches and connects it to OOD detection and prototype learning. Prior methods use decision scores, synthesized samples, or learned class representations, but some do not model deep unknown-class distributions.

  • Open Set Recognition: Earlier OSR research broadly divides into discriminative models and generative models.The paper describes OSR as practically valuable because testing may include unknown classes.
  • Discriminative Methods: Discriminative methods include SVM-based extreme-value models and extreme value machines that model class-inclusion probabilities.These approaches use decision-score distributions or extreme-value-theory density functions for unknown detection.
  • Generative Methods: Generative methods use GANs, autoencoders, or flow-based models to synthesize known or unknown samples for learning decision boundaries.Examples include G-Openmax and CROSR.
  • Open Set Recognition: Some prior methods do not consider deep distributions of unknown classes, leaving potential open space risk.This limitation is stated for the reviewed methods.
  • Out-of-Distribution Detection: OOD detection identifies test samples outside the training set and is related to OSR because both separate in-distribution known samples from out-of-distribution unknown samples.The paper also discusses maximum softmax probability, temperature scaling, input perturbations, generative models, and outlier exposure.
  • Prototype Learning: Prototype learning represents a category with an average or best exemplar, and neural-network prototypes are learnable latent vectors associated with classes.The related work discusses KNN, LVQ, center loss, and generalized convolutional prototype learning.

3 ADVERSARIAL RECIPROCAL POINT LEARNING

ARPL reframes open space risk through multiclass integration and reciprocal points, then constrains class-wise open spaces with an adversarial margin. Its reciprocal-point classification and margin losses jointly improve known-class separation while bounding unknown-space overlap.

  • Problem Definition: Open set recognition minimizes empirical classification risk on labeled known data and open space risk on potential unknown data simultaneously.The multiclass formulation integrates multiple one-versus-rest classification tasks and combines both risks over allowable recognition functions.
  • Reciprocal Points for Classification: Each reciprocal point represents the latent extra-class space for one known category, and samples outside that category should be closer to its reciprocal point than to the category.The framework classifies samples through opposition between known classes and their corresponding reciprocal points.
  • Reciprocal Points for Classification: Reciprocal-point classification assigns larger class probability to greater distance from that class’s reciprocal point while reducing empirical classification risk.The distance combines Euclidean and dot-product terms, allowing reciprocal points to oppose each known class in both spatial position and angle direction.
  • Adversarial Margin Constraint: The Adversarial Margin Constraint limits each known class’s distance to its reciprocal point by a learnable margin R, indirectly bounding open space risk.Because known and open spaces are complementary, constraining known samples below R restricts the corresponding open space; Euclidean distance is used to cover a larger range of non-k samples.
  • Adversarial Margin Constraint: Theorem 1 guarantees that simultaneous minimization of classification and margin losses constrains multiclass extraclass spaces into bounded regions.The theorem’s contradiction argument links the reciprocal-point distance constraint to preserving the intended class assignment and reducing open space risk.

4 INSTANTIATED ADVERSARIAL ENHANCEMENT

The instantiated adversarial enhancement generates confusing samples from open space and uses them with ARPL to improve discrimination between known and unknown data. The framework combines a discriminator, generator, classifier, reciprocal-point entropy objective, auxiliary normalization, and focus training.

  • Motivation: The method generates Confusing Samples as synthetic unknown data to improve classifier discriminability across novel distributions.These samples address the difficulty of finding valid unknown examples in real scenes.
  • Generator: The confused generator maps latent variables to samples that resemble known data while remaining close to all reciprocal points in embedding space.This adversarial objective places generated samples near the global open space and known-sample boundary.
  • Generator: The generator’s entropy objective is maximized when the distances from a generated feature to all reciprocal points are equal.Equal distances produce a uniform per-class softmax distribution over reciprocal points.
  • Classifier: The classifier is trained with generated confusing samples to estimate the unknown distribution of the global open space and reduce its size.Known and generated samples are processed as distinct distributions during this training.
  • Optimization: Auxiliary Batch Normalization isolates normalization statistics for confusing samples, while alternating optimization jointly updates the discriminator, generator, classifier, reciprocal points, and loss parameters.Focus Training is added after classifier training with confusing samples to improve attention to known-class classification.
  • Reliability Enhancement: The framework uses confrontation between known space and reciprocal-point open space, with a calibration-free score based on differences between known and unknown probabilities.The generated images cover low-response unknown feature regions and include confusing samples resembling known classes.

5.1 Implementation Details

Implementation fixes training hyperparameters, initializes reciprocal points and margins, and specifies the feature representation used for open set recognition and out-of-distribution experiments.

  • Implementation Details: γ is set to 1.0, while λ and β are set to 0.1 in all training phases and selected by cross-validation.Reciprocal points use random normal initialization, and each margin starts at one.
  • Implementation Details: For open set recognition, global average pooling follows the encoder’s final convolution layer before feature-based processing.Out-of-distribution experiments use the ResNet output after global average pooling as the feature.

5.2 Experiments for Open Set Recognition

The open set experiments evaluate randomized known–unknown protocols with AUROC and OSCR, emphasizing the trade-off between unknown detection and known-class classification. ARPL and ARPL+CS outperform the compared baselines across these evaluations.

  • Datasets: The protocols sample six known and four unknown classes for MNIST, SVHN, and CIFAR10; CIFAR+N uses four CIFAR10 known classes and N CIFAR100 unknown classes.TinyImageNet uses 20 known and 180 unknown classes.
  • Evaluation Metrics: AUROC is threshold-independent, while OSCR evaluates open set performance by combining correct known classification with unknown rejection across thresholds.Both metrics are averaged over five randomized trials.
  • Results: ARPL using only known training samples significantly outperforms most compared discriminative and generative OSR approaches.The method pushes known classes away from unknown classes through reciprocal points.
  • Results: ARPL+CS performs significantly better than recent generative methods and ARPL, especially on SVHN, CIFAR, and TinyImageNet.Confusing samples improve the reliability of the ARPL classifier.
  • Results: 3.6% improvement on TinyImageNet is reported for ARPL with confusing samples over ARPL under OSCR evaluation.The results are reported while balancing unknown detection with known-class classification.

5.3 Experiments for Out-of-Distribution Detection

The out-of-distribution experiments evaluate near and far OOD detection using CIFAR10 with CIFAR100 and SVHN, respectively. ARPL benefits from confusing samples, ABN, and FT, although near-OOD detection initially does not improve before FT.

  • Datasets and Metrics: CIFAR100 and SVHN serve as near-OOD and far-OOD datasets, respectively, for CIFAR10, with mutually exclusive CIFAR10 and CIFAR100 classes.Evaluation uses AUROC, TNR at 95% TPR, AUPR, and detection accuracy.
  • Results: ARPL without auxiliary confusing-sample training outperforms the generative OOD model JCL.The comparison attributes this advantage to modeling differences between unknown and known samples in deep feature space.
  • Ablation: ARPL+CS without ABN performs much worse than ARPL, supporting the assumption that confusing samples and known images have different underlying distributions.The adversarial mechanism progressively separates their distributions despite the generator’s initial similarity to known data.
  • Ablation: Near-OOD detection does not initially improve with confusing samples, possibly because they affect known-class discriminative features during training.This passage presents the explanation as a possibility rather than a demonstrated cause.
  • Ablation: Focus Training further improves both near- and far-OOD performance by directing the classifier’s attention toward known-class classification.The reported improvement follows the prior initialization of confusing samples.

5.4 Ablation Study

The ablations examine how reciprocal-point constraints, confusing samples, and training enhancements affect known classification and unknown detection. ARPL’s bounded open-space design and adversarial enhancements improve separation and stability across settings, while highly similar unknowns remain difficult.

  • ARPL vs. Softmax: ARPL’s open-space constraint separates known and unknown features and preserves high accuracy at low false-positive rates.The constraint bounds feature responses and pushes known classes away from global open space.
  • ARPL vs. GCPL: ARPL avoids GCPL’s initialization sensitivity by keeping known classes away from reciprocal unknown representations and maintaining a gap between sample types.GCPL extends unknown classes across the feature space, increasing overlap and open-space risk.
  • ARPL vs. RPL: ARPL improves over RPL through angular similarity estimation and adaptive regularization that reduce open-space risk in an elastic bounded space.Adaptive regularization avoids the instability and discriminability loss associated with overly strong fixed margins.
  • ARPL vs. ARPL + Confusing Samples: Confusing samples improve detection of diverse unknown categories while preserving known-class classification accuracy, with larger gains for far out-of-distribution data.They also improve detection of visually similar KMNIST samples despite limited feature-visualization changes.
  • Training Enhancements: ABN prevents confining samples from negatively affecting the classifier, while FT further enhances unknown-class discrimination during training.Without ABN, ARPL training collapses on MNIST; separate batch normalization protects known-class training from distribution differences.
  • Number of Known Classes: Increasing the number of known classes increases the margin, while ARPL retains discriminative feature distributions and unknown-detection performance.The analysis reports that the earlier ARPL conclusion holds across different numbers of known classes.
  • Margin Analysis: Open-set recognition remains difficult for unknown classes that closely resemble known classes, which can produce high feature responses under limited known priors.The authors identify these near-known unknowns as a future research challenge.

5.5 Further Analysis

Further experiments test ARPL on closed-set, distribution-shifted, large-scale, and class-incremental settings. ARPL remains competitive or superior across these settings, while reciprocal points substantially resist catastrophic forgetting as new classes arrive.

  • Closed-Set Recognition: ARPL achieves comparable closed-set performance to softmax and GCPL on CIFAR10, CIFAR100, and Air-300.Reciprocal points increase interclass distances and support more discriminative known-class features.
  • DomainNet: ARPL effectively detects semantic, nonsemantic, and combined distribution shifts on DomainNet, with confusing samples improving detection across domains.The reported difficulty trend is semantic shift > non-semantic shift > semantic + non-semantic shift.
  • ImageNet: ARPL outperforms softmax, GCPL, and RPL on ImageNet-1K/ImageNet-O for both close-set accuracy and AUROC.The comparison is conducted on a larger and more difficult dataset pairing ImageNet-1K as known data with ImageNet-O as unknown data.
  • ImageNet: Approximately 12% improvement over softmax is reported for unknown detection on the ImageNet evaluation.The authors attribute the separation to reciprocal-point constraints on global open space while preserving known-class recognition.
  • Class-Incremental Learning: Reciprocal points outperform alternative bases by a significant margin as new CIFAR100 classes are added and better resist catastrophic forgetting.The incremental experiment uses half the classes as base classes and introduces the remaining classes sequentially.

6 CONCLUSION

The paper introduces reciprocal points and ARPL to jointly address classification risk and open-space risk in open set recognition. Experiments report superior performance to prior state-of-the-art open-set classifiers, while the analysis identifies dependence on known priors as a remaining challenge.

  • Conclusion: Reciprocal points model the extraclass space corresponding to each known category, and ARPL uses adversarial margin constraints and confusing samples for open-set discrimination.The framework targets both empirical classification risk and open-space risk.
  • Conclusion: Extensive experiments on multiple datasets report that ARPL outperforms previous state-of-the-art open-set classifiers in all cases.
  • Conclusion: Unknown recognition is largely based on known priors, with unknown features concentrated in low-response regions and known features in high-response regions.The authors identify neural mechanisms of few-shot learning and improved unknown-category detection as future research directions.
Loading 2103.00953v3…