Source-linked AI summary
Classification-Reconstruction Learning for Open-Set Recognition
Ryota Yoshihashi, Wen Shao, Rei Kawakami, Shaodi You, Makoto Iida, Takeshi Naemura
TL;DR
Open-set recognition must distinguish unknown classes absent from training, but supervised representations emphasize known-class discrimination. CROSR jointly trains classification and reconstruction, using hierarchical latent representations for unknown detection; experiments show it outperforms prior deep open-set classifiers in most cases. The authors also note assumptions and evaluation choices that constrain interpretation.
Problem
Supervised deep open-set classifiers learn features emphasizing known classes, which may not represent unknowns or separate unknowns from knowns.
Method
CROSR jointly trains classification and reconstruction, using DHRNet hierarchical bottleneck representations together with predictions for unknown detection.
Results
CROSR outperforms previous state-of-the-art open-set classifiers in most cases across comprehensive experiments on multiple standard datasets.
Takeaways & Limitations
Reconstructive latent representations can complement prediction features in open-set recognition by supplying information useful for unknown detection.
Takeaways & Limitations
The Openmax comparison uses a simplified class-belongingness model depending only on y, and its hyperparameters were not tuned for each test set.
Abstract
from arXiv · showhide
Open-set classification is a problem of handling `unknown' classes that are not contained in the training dataset, whereas traditional classifiers assume that only known classes appear in the test environment. Existing open-set classifiers rely on deep networks trained in a supervised manner on known classes in the training set; this causes specialization of learned representations to known classes and makes it hard to distinguish unknowns from knowns. In contrast, we train networks for joint classification and reconstruction of input data. This enhances the learned representation so as to preserve information useful for separating unknowns from knowns, as well as to discriminate classes of knowns. Our novel Classification-Reconstruction learning for Open-Set Recognition (CROSR) utilizes latent representations for reconstruction and enables robust unknown detection without harming the known-class classification accuracy. Extensive experiments reveal that the proposed method outperforms existing deep open-set classifiers in multiple standard datasets and is robust to diverse outliers. The code is available in https://nae-lab.org/~rei/research/crosr/.
1. Introduction
Open-set recognition addresses the failure of closed-world classifiers when deployment includes classes absent from training. CROSR combines supervised classification with reconstruction-based latent representations so known classification and unknown detection can use complementary information.
- Closed-set classifiers assume all deployment classes appear in training, but real-world coverage is easily violated by unknown classes.
- Existing deep open-set classifiers rely on supervised features specialized for known classes, which may not represent unknowns or separate them from knowns.
- The framework adds reconstruction of input samples to supervised classification, acquiring information that may be useful beyond discriminating the given classes.
- CROSR jointly learns known-class predictions y and reconstructive latent representations z, using y for classification and both y and z for unknown detection.
- DHRNets learn hierarchical bottleneck representations from intermediate layers, preserving multilevel anomaly factors while maintaining compact spaces for unknown detection.
- Experiments on five standard image and text datasets show CROSR outperforms existing deep open-set classifiers for most combinations of known data and outliers.
2. Related work
Related work spans classical open-set classifiers, anomaly detectors, deep open-set methods, and reconstruction-based semi-supervised learning. CROSR differs by integrating reconstructive representations with discriminative known-class classification for open-set recognition.
- Open-set classification: Classical open-set methods include Weibull-calibrated SVMs, similarity-space models, and extreme value machines for modeling class inclusion or decision-score distributions.
- Open-set classification: Deep open-set classifiers include G-Openmax, which synthesizes unknown data but is limited beyond handwritten characters, and DOC, designed for document classification.
- Outlier detection: Generic outlier detectors lack open-set classification capability because they detect anomalies without discriminating among known classes.
- Outlier detection: Recent anomaly-detection approaches use autoencoders, Gaussian mixtures, clustering, one-class learning, or generative adversarial networks, but their usage differs from CROSR's latent-representation approach.
- Outlier detection: Supervised anomaly methods using input preprocessing cannot be simply extended to open-set classifiers because preprocessing may degrade known-class classification.
- Semi-supervised learning: Reconstruction serves as a data-dependent regularizer in semi-supervised learning and domain adaptation; CROSR demonstrates its use in open-set classification.
3. Preliminaries
Openmax extends Softmax for open-set recognition by calibrating final-layer activations with class-belongingness probabilities. Its formulation models class affinity using distances and Weibull distributions, while simplifying those probabilities to depend only on the final representation.
- Openmax extends Softmax by adding open-set handling to a classifier defined over known classes and an input x.
- The network maps x to a final hidden-layer representation y, also called the activation vector, whose dimensionality equals the number of known classes.
- Open-set recognition calibrates the activation vector using class-inclusion probabilities so outliers receive reduced confidence for known classes.
- Openmax represents the unknown category as class C_N+1 and uses p(x ∈ C_i) as a proxy for the harder-to-model probability p(x ∈ K).
- Class-belongingness is modeled from distances to class representations using Weibull distributions and parameters derived from training data.
- The method uses an ℓ2 distance between activation vectors and class means, with a heuristic calibrator controlled by α and activation-vector rank.
- The formulation makes the simplifying assumption that class-belongingness p(x ∈ C_i) depends only on y.
4. CROSR: Classification-reconstruction learning for open-set recognition
CROSR combines supervised known-class prediction with reconstructive latent representations to improve unknown detection. Its DHRNet architecture uses bottlenecked lateral connections and hierarchical representations while preserving classification features.
- Open-set classification with latent representations: CROSR jointly uses the prediction y and reconstructive latent representation z to model class belongingness, rather than relying on activation vectors alone.The latent representation encodes more information about the input and is concatenated with y for unknown detection.
- Deep hierarchical reconstruction nets: DHRNets use bottlenecked lateral connections to learn rich representations for classification and compact representations for unknown detection jointly.They reconstruct intermediate classification layers through low-dimensional latent representations, producing hierarchical bottlenecks.
- Deep hierarchical reconstruction nets: DHRNet extracts bottlenecks z1, z2, z3, ..., zL from multistage features and concatenates them into the unknown detector input.This design targets outlying factors that may disappear at intermediate stages of the inference chain.
- Deep hierarchical reconstruction nets: Unlike classification-only networks and standard autoencoders, DHRNet preserves classification capacity while introducing compressed lateral representations for reconstruction.Classification-only models lack useful outlier-detection factors beyond y, whereas autoencoders can compress intermediate layers too aggressively for large-scale classification.
- Deep hierarchical reconstruction nets: DHRNet reconstructs each intermediate layer by combining top-down decoded information with reprojected lateral bottleneck information.The architecture uses dimensionality reduction and reprojection around each bottleneck, while the decoder combines the two information streams.
- Training: Training minimizes classification and reconstruction errors on known-class data, using softmax cross entropy for labels and modality-specific reconstruction losses.Image reconstruction uses ℓ2 distance, while text reconstruction uses cross entropy of one-hot word representations; unknown-class data are unavailable during training.
5. Experiments
Experiments evaluate CROSR across image and text datasets, class-separation and outlier-addition protocols, and diverse unknown sets. CROSR generally improves open-set detection over deep baselines while reconstruction-based training preserves known-class classification accuracy.
- Experimental setup: Experiments use five standard datasets and evaluate both class separation and outlier addition, treating all unknown labels as one unknown class.Class separation uses held-out classes from the same domain, while outlier addition introduces samples from another dataset during testing.
- Accuracy and robustness: Adding reconstruction terms caused no significant degradation in known-class accuracy, while CROSR improved unknown detection across most evaluated dataset–outlier combinations.Visualization also shows CROSR assigning lower confidence to unknown samples, with deception concentrated among samples resembling inliers.
- Experimental setup: MNIST experiments add Omniglot, Noise, and MNIST-Noise outliers, with MNIST-Noise designed to resemble known samples more closely.Each outlier set contains 10,000 test images, producing a known-to-unknown ratio of 1:1.
- Open-set results: CROSR consistently outperformed competing methods except in two settings, exceeding Supervised + Openmax by more than 10% in MNIST F1-score for Omniglot and MNIST-Noise outliers.It slightly underperformed on the easiest Noise outliers and outperformed or matched LadderNet + Openmax and DHRNet + Openmax.
- Open-set results: CROSR outperformed other methods on DBpedia except the 4/4 train/test-class setting, while its improvement over Openmax was significant.The 4/4 setting is equivalent to closed-set classification.
- Comparison with GAN-based methods: CROSR outperformed all GAN-based methods on MNIST and TinyImageNet and all except Counterfactual on SVHN, without synthesized training data.The reported relative improvements were within the error bars, indicating performance on par with or slightly better than those methods.
6. Conclusion
CROSR combines latent-representation reconstruction with open-set classification and adds a deep hierarchical reconstruction architecture for unknown detection. Experiments on multiple standard datasets showed that it outperformed previous state-of-the-art open-set classifiers in most cases.
- CROSR augments deep open-set classification with latent representation learning for reconstruction.
- Deep hierarchical reconstruction nets provide the architecture developed to improve the usability of latent representations for unknown detection.
- Experiments on multiple standard datasets showed that CROSR outperformed previous state-of-the-art open-set classifiers in most cases.