Source-linked AI summary
Data Augmentation Generative Adversarial Networks
Antreas Antoniou, Amos Storkey, Harrison Edwards
TL;DR
Low-data neural networks need broader plausible augmentations than standard transformations provide. The paper introduces DAGAN, which learns class-agnostic within-class augmentations from a source domain and applies them to target data. DAGAN improves vanilla classifiers and few-shot matching networks across Omniglot, EMNIST, and VGG-Face experiments.
Problem
Standard data augmentation uses a limited set of known invariances, while low-data learning requires more effective use of existing examples.
Method
DAGAN is a conditional GAN trained in a source domain to transform any input into varied within-class examples without relying on class identities.
Results
DAGAN improves classifier performance across Omniglot, EMNIST, and VGG-Face and enhances matching networks in Omniglot and EMNIST.
Takeaways & Limitations
DAGAN provides a flexible augmentation approach for low-data settings and can improve classifiers after standard augmentation and meta-learning systems.
Takeaways & Limitations
The augmentation model assumes latent variation can be represented using a Gaussian variable.
Abstract
from arXiv · showhide
Effective training of neural networks requires much data. In the low-data regime, parameters are underdetermined, and learnt networks generalise poorly. Data Augmentation alleviates this by using existing data more effectively. However standard data augmentation produces only limited plausible alternative data. Given there is potential to generate a much broader set of augmentations, we design and train a generative model to do data augmentation. The model, based on image conditional Generative Adversarial Networks, takes data from a source domain and learns to take any data item and generalise it to generate other within-class data items. As this generative process does not depend on the classes themselves, it can be applied to novel unseen classes of data. We show that a Data Augmentation Generative Adversarial Network (DAGAN) augments standard vanilla classifiers well. We also show a DAGAN can enhance few-shot learning systems such as Matching Networks. We demonstrate these approaches on Omniglot, on EMNIST having learnt the DAGAN on Omniglot, and VGG-Face data. In our experiments we can see over 13% increase in accuracy in the low-data regime experiments in Omniglot (from 69% to 82%), EMNIST (73.9% to 76%) and VGG-Face (4.5% to 12%); in Matching Networks for Omniglot we observe an increase of 0.5% (from 96.9% to 97.4%) and an increase of 1.8% in EMNIST (from 59.5% to 61.3%).
1 INTRODUCTION
DAGAN learns broader data augmentations from a source domain to improve classifiers in low-data target domains, including unseen classes and few-shot systems.
- Deep neural networks generally require very large datasets or extensive experience to achieve strong performance.
- Standard augmentation applies limited known invariances, whereas DAGAN learns a larger invariance space in a source domain for use in a target domain.
- DAGAN does not depend on class identities, allowing its learned augmentation process to transfer to novel classes.
- The paper reports state-of-the-art meta-learning performance through novel data-augmentation strategies.
2 BACKGROUND
The paper frames one-shot learning as extreme dataset shift and contrasts fixed, known-invariance augmentation with learned transformations that can transfer across related problems.
- Transfer Learning and Dataset Shift: One-shot learning changes the class distribution dramatically, while assuming shared class-conditional structure permits transfer from source to target domains.
- Generative Adversarial Networks: GANs learn complex joint densities by discriminating true examples from generated examples, with improved optimization reducing some failure modes.
- Data Augmentation: Conventional augmentation encodes known invariances through transformations such as shifts, rotations, shears, intensity changes, and stroke changes.
- Data Augmentation: Earlier learned augmentation strategies were class-specific and therefore did not transfer to one-shot settings with completely new classes.
- Few-Shot Learning and Meta-Learning: Few-shot and meta-learning research includes generative, Siamese, Bayesian, metric-learning, memory-augmented, and related neural approaches.
3 MODELS FOR DATA AUGMENTATION
DAGAN learns a conditional augmentation manifold that preserves relationships to an input while generating varied within-class examples without using class labels.
- The proposed model combines an input representation with random variation to generate additional data related to an observed example.
- A GAN can map a data manifold by transforming latent Gaussian variables into generated samples whose distribution matches the training data.
- For a new input, DAGAN encodes a representation and samples Gaussian noise to generate augmentation examples for classifier training.
- Learning: The critic compares real same-class pairs with generator outputs, while the generator minimizes the critic’s Wasserstein discrimination.
- Learning: Conditioning the discriminator on the original input discourages autoencoding, while omitting class information encourages transformations consistent across classes.
4 ARCHITECTURES
The main DAGAN generator uses a UResNet architecture combining UNet and ResNet components with repeated convolutional, downscaling, and upscaling blocks.
- The main experiments use a UResNet generator that combines UNet and ResNet designs.
- The generator contains eight blocks, each with four convolutional layers followed by a downscaling or upscaling layer.
- Its blocks use leaky ReLU activations, batch renormalisation, convolutions, and dropout in the downscaling path.
5 DATASETS
The study evaluates DAGAN augmentation across Omniglot, EMNIST, and VGG-Faces, using source, validation, and test domain splits for experiments.
- DAGAN augmentation was tested on Omniglot, EMNIST, and VGG-Faces, with each dataset randomly divided into source, validation, and test domain sets.
- For classifier experiments, each character or person was further split into two test cases, three validation cases, and a variable number of training cases.
- For one-shot experiments, DAGAN and meta-learning were trained on source domains, validated on validation domains, and evaluated on target-domain test data.
6 DAGAN TRAINING AND GENERATION
DAGAN training compared generator architectures and used generated examples to augment vanilla classifiers. More powerful architectures improved generation variability, while augmentation improved classifier performance in every reported case.
- DAGAN training and generation: DAGANs trained with VGG, U-net, and ResNet-inspired architectures showed improved generation variability as architectures became more powerful, leading to selection of a UResNet generator.
- Vanilla classifiers: The vanilla-classifier experiment compared DenseNet training with standard augmentation alone against training that also included DAGAN-generated data and real-or-fake labels.
- DAGAN training and generation: DAGAN generation can augment a single real seed image with additional generated examples arranged across an interpolated spherical subspace of the GAN generation space.
- Vanilla classifiers: DAGAN augmentation improves classifier performance in all cases reported in Table 1, whose test accuracy is measured on test cases in the test domain.
7 ONE SHOT LEARNING USING DATA AUGMENTATION NETWORKS AND MATCHING NETWORKS
The paper applies DAGAN augmentation to Matching Networks for one-shot classification, adding generated images during training episodes. On Omniglot, this improves both simple pixel-distance classification and Matching Network performance.
- Matching Networks: Matching Networks learn a representation space in which distances support nearest-neighbour classification and can then be applied to a target domain.
- DAGAN augmentation: DAGAN augmentation was used during every Matching Network training episode, with generated images stacked alongside the original image.
- Omniglot results: 33.815% improvement was observed for simple pixel-distance nearest-neighbour classification with DAGAN augmentation in Omniglot one-shot experiments.
- Omniglot results: 0.5% improvement raised Matching Network performance from 96.9% to 97.4%, reaching the level of Conv-ARC in the Omniglot experiments.
8 CONCLUSIONS
The paper concludes that DAGAN is a flexible approach for automatically learning data augmentation and improves classifiers and meta-learning methods in low-data settings.
- DAGAN improves classifier performance even after standard data augmentation and, through meta-learning, outperforms other state-of-the-art meta-learning methods.
- The authors describe data augmentation as widely applicable to low-data settings and DAGAN as potentially useful across models and methods.
A APPENDIX
The appendix presents the UResNet generator’s visual form and detailed architecture, including its notation and latent-sampling step.
- The appendix provides a drawing of the UResNet generator corresponding to Algorithm 1.
- Algorithm 1 is titled “U-ResNet Generator Architecture” and defines notation for encoder, decoder, layer, scaling, and projection components.
- The architecture procedure samples a 100-dimensional vector from N(0, 1) before projecting it to match the required dimensionality.