Source-linked AI summary
Generative Adversarial Residual Pairwise Networks for One Shot Learning
Akshay Mehrotra, Ambedkar Dukkipati
TL;DR
One-shot learning remains challenging in the low-data regime, and fixed similarity measures may limit the objective a model can learn. The paper proposes a residual pairwise network with end-to-end trainable similarity and a GAN-based regularizer. It reports state-of-the-art performance on mini-Imagenet one-shot classification, while noting challenges in training the generator–discriminator framework.
Problem
Method
The paper combines an end-to-end trainable residual pairwise similarity network with a GAN-based generator whose discriminator is the pairwise network.
Source: §1 · pp. 1–2, §3.2 · p. 4Results
The Skip Residual Pairwise Network achieves state-of-the-art performance on the mini-Imagenet one-shot classification dataset.
Source: §6 · p. 7Takeaways & Limitations
The Generative Regularizer shows promising results and outperforms L2 regularization on Omniglot.
Source: §6 · p. 7GAN training is sensitive to hyperparameters and can produce poorly trained generators that impede training, particularly with very deep discriminators.
Source: §5 · p. 7
Abstract
from arXiv · showhide
Deep neural networks achieve unprecedented performance levels over many tasks and scale well with large quantities of data, but performance in the low-data regime and tasks like one shot learning still lags behind. While recent work suggests many hypotheses from better optimization to more complicated network structures, in this work we hypothesize that having a learnable and more expressive similarity objective is an essential missing component. Towards overcoming that, we propose a network design inspired by deep residual networks that allows the efficient computation of this more expressive pairwise similarity objective. Further, we argue that regularization is key in learning with small amounts of data, and propose an additional generator network based on the Generative Adversarial Networks where the discriminator is our residual pairwise network. This provides a strong regularizer by leveraging the generated data samples. The proposed model can generate plausible variations of exemplars over unseen classes and outperforms strong discriminative baselines for few shot classification tasks. Notably, our residual pairwise network design outperforms previous state-of-theart on the challenging mini-Imagenet dataset for one shot learning by getting over 55% accuracy for the 5-way classification task over unseen classes.
1. Introduction
The paper targets one-shot classification of unseen classes, proposing an end-to-end trainable pairwise similarity model and GAN-based regularization. It situates these ideas within similarity matching and generative approaches, then identifies them as its two contributions.
1. Introduction
- One-shot learning tests whether a model can classify unseen classes from a single example per class.The paper frames the task as learning features that generalize across classes.Source: §1 · p. 1
- The proposed Skip Residual Pairwise Network (SRPN) learns a distance measure end-to-end using a residual-network modification and is evaluated on Omniglot and mini-Imagenet.Unlike a fixed distance function applied to embeddings, its pairwise network takes two data points and optimizes their similarity directly.Source: §1 · p. 1, Figure 1 · p. 2
- The Generative Regularizer uses a GAN framework to generate conditional data that regularizes a siamese matching network and supports generalization to unseen classes.The discriminator incorporates feedback that trains the generator alongside similarity matching, making the model an instance of multi-task learning.Source: §1 · pp. 1–2
1.1. Related Work
- The paper places its work in similarity matching and contrasts pairwise comparisons with full-context and optimization-based few-shot approaches [30] [34].Related work also covers Bayesian methods [7], metric learning [13] [24], and generative models, including GAN-based approaches [21] [27].Source: §1.1 · p. 2
1.2. Contributions
- The contributions are a trainable distance measure that achieves state-of-the-art performance on mini-Imagenet and a GAN-based network using generated data as a regularizer for similarity matching.The paper reports improvements for one-shot learning from the GAN-based design.Source: §1.2 · p. 3
2. Preliminaries
The preliminaries define few-shot evaluation on classes disjoint from training and describe GANs as generator–discriminator models trained through a min-max objective.
2.1. Few Shot Learning
- Few-shot evaluation uses a support set with one or five examples per class from test classes disjoint from training, then asks the model to classify another test sample.The number of test classes, generally five or 20, and examples per class characterize the task.Source: §2.1 · p. 3
2.2. Generative Adversarial Networks
- GANs use a generator mapping latent vectors to data and a discriminator estimating whether input data came from the real-data distribution; training is formulated as min-max optimization.The generator is trained to maximize discriminator error on generated samples.Source: §2.2 · p. 3
3. Models
The models replace fixed similarity over separate embeddings with a learned pairwise network, then add a generator to regularize similarity matching with generated examples. The pairwise network mixes information across the two inputs while retaining residual pathways; the generator produces plausible same-class variations for the discriminator.
3.1. Skip Residual Pairwise Network
- The pairwise network learns similarity end-to-end from paired inputs, avoiding a fixed function over separately generated embeddings.Its pathways alternate residual skip connections with convolutions after splitting the intermediate representations.Source: §3.1 · p. 3
- The pairwise pathways mix information between inputs while preserving residual structure, unlike standard Siamese networks that mix only at the final layer.The final output is a single embedding for a linear similarity classifier trained with binary cross-entropy.Source: §3.1 · p. 3
3.2. Generative Regularizer
- The discriminator predicts whether an input belongs to the same class-conditional distribution as its comparison image and separately identifies generated images as fake.The design avoids explicit class labels to discourage memorizing class details and allow conditioning on unseen classes.Source: §3.2 · p. 4
- The generator corrupts a comparison image and maps it to a plausible variation, using an adversarial objective rather than a reconstruction loss.It acts as a denoising autoencoder, with down-scaling and transposed-convolution up-scaling.Source: §3.2 · pp. 3–4
4. Experiments
The experiments evaluate Skip Residual Pairwise Networks (SRPN) and Generative Regularization (GR) on Omniglot and mini-Imagenet few-shot tasks. On mini-Imagenet, GR did not benefit the deep models, so only SRPN results are reported.
4.1. Omniglot
- Omniglot experiments test both SRPN and GR using 1,200 training/validation classes and the remaining classes for few-shot testing.The protocol uses 200 tests with 20 runs per test; the study follows the across-alphabet setting.Source: §4.1 · pp. 4–5
4.2. Mini-Imagenet
- On mini-Imagenet, GR did not benefit the deep models, and the authors report only SRPN results, attributing the outcome to unstable GAN training.They report that the generator was not able to learn well for these very deep models.Source: §4.2 · p. 6
- Mini-Imagenet experiments use the first 80 classes for training and validation and the remaining classes for testing, with 100 tests and 100 runs per test.The dataset uses randomly selected 100-class ImageNet subsets; images are rescaled to 84x84 pixels without augmentation or preprocessing.Source: §4.2 · p. 6
4. Experiments
- The reported tables cover Omniglot 20-way one-shot accuracy and mini-Imagenet 5-way few-shot accuracy.The supplied table captions identify these evaluation scopes but do not provide the table values.Source: Table 1 · p. 6, Table 2 · p. 6
5. Discussion
The SRPN is reported to outperform the equivalent residual Siamese network, converge faster, and use smaller mean weights. The discussion attributes its performance to adapting the distance measure, while noting mixed results and training challenges for the Generative Regularizer.
5. Discussion
- The SRPN achieves higher accuracy than the equivalent Residual Siamese Network, converges faster, and has a noticeably smaller mean network weight.Figures 6 and 7 track mean-weight ratios and one-shot test accuracy during mini-Imagenet training.Source: §5 · p. 6, Figure 7 · p. 7, Figure 6 · p. 7
- The SRPN can adapt its distance measure to minimize total loss, which the authors connect to a manifold that reduces similarity loss and regularization penalty.They report that the model's embeddings become increasingly symmetric despite not being explicitly trained for symmetry.Source: §5 · p. 7, Figure 8 · p. 7
- The Generative Regularizer is more effective than L2 regularization on Omniglot with the Siam-I model, but its overall results are mixed.Source: §5 · p. 7
- Training the generator-discriminator framework is sensitive to hyperparameters, and poorly trained generators can impede training with very deep discriminators.Source: §5 · p. 7
- Without conditioning information, the generator collapsed to a single point yet still regularized the Siamese network.The authors take this as evidence that good generation is not essential for regularization.Source: §5 · p. 7
6. Conclusion
The conclusion identifies fixed distance measures and weak regularization as challenges for similarity matching and one-shot learning. It presents the SRPN and Generative Regularizer as solutions, reporting state-of-the-art mini-Imagenet performance for the former and results above L2 regularization on Omniglot for the latter.
6. Conclusion
- The paper identifies fixed distance measures and weak regularization as challenges, then reports state-of-the-art mini-Imagenet one-shot performance for SRPN and better-than-L2 results on Omniglot for its Generative Regularizer.Source: §6 · p. 7