Source-linked AI summary
Robust Classification with Convolutional Prototype Learning
Hong-Ming Yang, Xu-Yao Zhang, Fei Yin, Cheng-Lin Liu
TL;DR
CNNs achieve high accuracy but can be vulnerable to adversarial perturbations and confidently accept unseen classes. The paper proposes convolutional prototype learning with prototype loss, and reports comparable or better accuracy plus strong rejection and class-incremental robustness.
Problem
CNNs’ softmax-based discriminative formulation has limited robustness for adversarial examples and unseen-class rejection.
Method
CPL jointly learns CNN features and multiple class prototypes, uses nearest-prototype matching, and adds prototype loss to compact within-class representations.
Results
CPL achieves comparable or better classification accuracy than traditional CNNs, while GCPL rejects over 99% of CIFAR-10 samples and accepts over 99% of MNIST samples.
Takeaways & Limitations
Prototype-based decision making gives GCPL advantages in outlier rejection and class-incremental learning.
Takeaways & Limitations
The authors state that accuracy is not intended to be significantly better than previous softmax-based CNNs, and further evaluation of other GCPL properties remains future work.
Abstract
from arXiv · showhide
Convolutional neural networks (CNNs) have been widely used for image classification. Despite its high accuracies, CNN has been shown to be easily fooled by some adversarial examples, indicating that CNN is not robust enough for pattern classification. In this paper, we argue that the lack of robustness for CNN is caused by the softmax layer, which is a totally discriminative model and based on the assumption of closed world (i.e., with a fixed number of categories). To improve the robustness, we propose a novel learning framework called convolutional prototype learning (CPL). The advantage of using prototypes is that it can well handle the open world recognition problem and therefore improve the robustness. Under the framework of CPL, we design multiple classification criteria to train the network. Moreover, a prototype loss (PL) is proposed as a regularization to improve the intra-class compactness of the feature representation, which can be viewed as a generative model based on the Gaussian assumption of different classes. Experiments on several datasets demonstrate that CPL can achieve comparable or even better results than traditional CNN, and from the robustness perspective, CPL shows great advantages for both the rejection and incremental category learning tasks.
1. Introduction
The paper attributes CNNs’ limited robustness to adversarial examples and unseen-class rejection to discriminative softmax classification and weak feature compactness. It proposes convolutional prototype learning, enhanced with prototype loss, to improve representation robustness while preserving classification accuracy.
- Motivation: CNNs can misclassify barely altered adversarial samples with high confidence, revealing limited robustness despite strong accuracy.The paper also identifies poor rejection of samples from unseen classes as a related robustness problem.
- Motivation: Traditional CNN representations can have smaller inter-class than intra-class variation, reducing robustness in complex real-world scenes.Figure 1 illustrates this feature-space issue on MNIST.
- Motivation: A discriminative CNN partitions the whole feature space, so unseen samples fall into known-class regions and receive confident predictions.The paper links this mechanism specifically to CNNs’ poor rejection ability.
- Convolutional prototype learning: CPL replaces softmax classification with multiple learned prototypes per class and assigns samples to the nearest prototype in feature space.The CNN feature extractor and prototypes are learned jointly from raw data using multiple classification criteria.
- Prototype loss: Prototype loss pulls features toward corresponding prototypes, improving intra-class compactness and producing a hybrid discriminative-generative representation.The paper interprets prototypes as Gaussian-component means and describes the resulting representation as more discriminative and robust.
2. Related works
The related-work discussion places CPL at the intersection of robust representation learning and prototype-based classification. The paper positions its contribution as combining learned prototype classifiers with deep convolutional feature extraction.
- Robust representation learning: Recent robust-representation methods modify loss functions, including contrastive and triplet losses, to shape CNN features.These methods may require carefully selected input pairs or triplets for convergence and stability.
- Prototype learning: Prototype learning includes K-NN, LVQ, and optimization-based methods that learn prototypes through updating rules or loss functions.Earlier approaches mainly used hand-designed features before CNN-based prototype learning.
- Combined framework: The paper describes CPL as combining prototype-based classifiers with deep convolutional neural networks for high-accuracy and robust pattern classification.Figure 2 provides an illustration of the CPL framework.
3. Convolutional prototype learning
CPL replaces softmax classification with jointly learned CNN features and class prototypes, using prototype matching and several distance-based losses. Prototype loss further regularizes the representation by improving intra-class compactness and supporting more robust open-set recognition.
- Architecture: CPL uses CNN feature extraction followed by multiple learned prototypes per class, assigning each sample to the class of its nearest prototype.Prototype matching uses Euclidean distance in feature space, and the CNN extractor and prototypes are trained jointly from raw data.
- Architecture: The framework trains CNN parameters and prototypes jointly end-to-end, enabling the representation and prototype set to cooperate during classification.The trainable parameters are the CNN parameters θ and the class prototypes M.
- Classification losses: MCE-based training compares distances to the closest genuine-class and competing-class prototypes, encouraging samples toward their class and away from incorrect classes.The loss is zero for correctly classified samples under the stated distance ordering and active when the sample is misclassified.
- Classification losses: MCL adds a positive margin to penalize even correctly classified samples within that margin, while GMCL avoids selecting a margin on the unknown distance-difference scale.The paper states that GMCL addresses the need to estimate the scale of d(f(x), myi) − d(f(x), mrj) from training data.
- Classification losses: DCE defines class probabilities from distances to prototypes and minimizes distance to prototypes belonging to the sample’s genuine class.Unlike MCL and GMCL, DCE updates all prototypes during each training step and therefore converges faster.
- Prototype loss: Prototype loss regularizes CPL by pulling features toward corresponding prototypes, improving intra-class compactness and producing intra-class compact, inter-class separable representations.The paper presents PL as a generative-model-like regularizer intended to improve generalization and reduce over-fitting.
4. Application of GCPL
GCPL is applied to rejection and class-incremental learning. The framework uses prototype-based representations and existing rejection strategies, while new-class features can be clustered or summarized by prototypes without discarding old-class accuracy.
- Rejection and class-incremental learning: GCPL supports rejection and class-incremental learning, although the paper applies existing methods rather than introducing new ones for these tasks.Classification probabilities can still be obtained in GCPL, allowing softmax-based rejection strategies to be used.
- Rejection and class-incremental learning: For a new class, compact GCPL features can be clustered or averaged into prototypes while preserving recognition accuracy on previously learned classes.The stated class-incremental setting requires recognizing the new class correctly and keeping accuracy on old classes.
5. Experiments
Experiments evaluate CPL and GCPL against softmax-based CNNs across standard classification, rejection, class-incremental learning, small-sample, and multiple-prototype settings. GCPL generally matches or outperforms traditional CNNs while showing stronger robustness in rejection, incremental learning, and reduced-data conditions.
- Classification accuracy: CPL achieves comparable or better accuracy than softmax-based classification across different CIFAR-10 network structures, while GCPL performs best in all CIFAR-10 experiments.These results support the efficiency and generality of the framework across architectures.
- Classification accuracy: GCPL also performs well or better on the large-scale OLHWDB classification problem, demonstrating efficiency and generality beyond smaller benchmarks.OLHWDB contains 2,697,673 training samples, 224,590 test samples, and 3,755 classes.
- Rejection: Over 99% of CIFAR-10 outliers are rejected while over 99% of MNIST samples remain accepted, unlike softmax models’ incompatible acceptance and rejection rates.Acceptance rate measures accepted MNIST samples; rejection rate measures rejected CIFAR-10 samples.
- Class-incremental learning: GCPL supports class-incremental learning by adding a new class prototype from the new class’s training-data mean without retraining the existing model.The learned representations for known and unknown classes are described as intra-class compact and inter-class separable.
- Small sample size: With fewer training samples, GCPL accuracy declines more slowly and remains more stable than softmax-based CNN accuracy, including at 5% and 3% of the training data.GCPL achieves higher accuracy and smaller variances under reduced sample sizes.
- Multiple prototypes: Using more prototypes does not improve results in the tested setting, although additional prototypes may help when transformed data distributions are difficult to model.The authors report that a single Gaussian distribution, represented by one prototype, can model the post-CNN class distributions in this setting.
6. Conclusion
The paper proposes CPL, which replaces softmax-based decisions with learned prototype matching, and GCPL, which adds prototype loss to improve robustness. GCPL maintains competitive classification while offering advantages in outlier rejection and class-incremental learning.
- CPL learns multiple convolutional feature-space prototypes per class and uses prototype matching for decision making instead of softmax.
- CPL achieves comparable or better classification accuracy than softmax-based CNN models.
- Prototype loss improves intra-class compactness, producing the generalized CPL model, GCPL.
- GCPL has advantages over traditional CNN models for outlier rejection and class-incremental learning.