PaperlayerView exact revision on arXiv

Source-linked AI summary

FIGR: Few-shot Image Generation with Reptile

Louis Clouâtre, Marc Demers

arXiv:1901.02199v1cs.LGcs.CVstat.ML

TL;DR

GANs can generate realistic images, but their data requirements limit their usefulness for generating novelty. FIGR meta-trains a GAN with Reptile, and generates images from unseen MNIST and Omniglot classes with as few as 4 samples.

  • Problem

    GANs require far more data than humans to generate comprehensible images, limiting their usefulness for generating novelty.

    Source: §1 · p. 1

  • Method

    FIGR uses Reptile meta-training to train a GAN for few-shot image generation.

    Source: §1 · p. 1

  • Results

    4 samples from an unseen class suffice for FIGR to generate images on MNIST and Omniglot, while the authors describe FIGR-8 results as preliminary.

    Source: §6 · p. 6

  • Takeaways & Limitations

    The results demonstrate the potential of using Reptile to train GANs for few-shot image generation.

    Source: §6 · p. 6

    Omniglot classes are simple handwritten characters, and prior few-shot generators do not use architectures with GANs’ potential for highly realistic images.

    Source: §2.3 · p. 2, §2.2 · p. 2

Abstract

from arXiv · show

Generative Adversarial Networks (GAN) boast impressive capacity to generate realistic images. However, like much of the field of deep learning, they require an inordinate amount of data to produce results, thereby limiting their usefulness in generating novelty. In the same vein, recent advances in meta-learning have opened the door to many few-shot learning applications. In the present work, we propose Few-shot Image Generation using Reptile (FIGR), a GAN meta-trained with Reptile. Our model successfully generates novel images on both MNIST and Omniglot with as little as 4 images from an unseen class. We further contribute FIGR-8, a new dataset for few-shot image generation, which contains 1,548,944 icons categorized in over 18,409 classes. Trained on FIGR-8, initial results show that our model can generalize to more advanced concepts (such as "bird" and "knife") from as few as 8 samples from a previously unseen class of images and as little as 10 training steps through those 8 images. This work demonstrates the potential of training a GAN for few-shot image generation and aims to set a new benchmark for future work in the domain.

1. Introduction

GANs can generate realistic images but require far more data than humans, limiting their usefulness for generating novelty. FIGR applies Reptile meta-training to GANs for few-shot image generation and contributes FIGR-8 as a benchmark dataset.

1. Introduction

  • GANs require several orders of magnitude more data than humans to generate comprehensible class images, which impairs their ability to generate novelty.When data is abundant enough to train a GAN successfully, the authors argue there may be little purpose in generating more of that data.Source: §1 · p. 1
  • FIGR introduces Reptile-based GAN training for few-shot image generation, with reported generation from an unseen class using as few as 4 samples.The approach targets novelty by adapting image generation to a previously unseen class from a small number of examples.Source: §1 · p. 1
  • FIGR-8 contributes 1,548,944 pictograms and icons categorized into 18,409 classes as a benchmark for one- and few-shot image generation.The dataset includes black-and-white pictograms, ideograms, icons, emoticons, and depictions of objects or concepts.Source: §1 · p. 1
  • The authors foresee creative assistance as an application, with artists or designers sketching a few drawings to generate similar versions.The proposed use is to help create multiple image variants when time or creative inspiration is limited.Source: §1 · p. 1

2. Related work

Prior few-shot generation methods use approaches such as Bayesian programs, sequential generative models, and matching networks, while the authors identify gaps in scalability, image realism, and benchmark breadth. Omniglot is a common benchmark, but its character-focused classes are similar and simple.

2.1. Meta-learning

  • MAML-style meta-learning algorithms rely on loss direction reflecting model quality and on test-set evaluation, assumptions the authors say do not fit GAN meta-training.The authors report no clear way to evaluate GAN performance on a test set for training.Source: §2.1 · p. 1

2.2. Few-Shot Image Generation

  • Bayesian Program Learning generates novel binary samples of an unseen letter from one image, while sequential generative models and matching networks support one-shot and few-shot Omniglot image generation, respectively.Rezende et al. (2016) generate 28 × 28 and 52 × 52 Omniglot images with one-shot learning; Bartunov and Vetrov (2018) generate 28 × 28 images with few-shot learning.Source: §2.2 · p. 2
  • The authors identify the use of small binary images as a potential scalability issue and note that prior approaches do not use architectures with GANs’ demonstrated potential for highly realistic images.They also cite limitations in Omniglot as a benchmark for one- and few-shot image generation.Source: §2.2 · p. 2

2.3. Omniglot

  • Omniglot classes are similar handwritten characters with limited complexity, motivating a benchmark with greater class variety and complexity for potential real-life or natural-image applications.The authors describe Omniglot as the current baseline dataset for one- or few-shot image generation.Source: §2.3 · p. 2

3. Few-shot Image Generation with Reptile

FIGR frames few-shot image generation as adapting a GAN to a new image class from a small sample, with performance defined by human discrimination. Reptile meta-training moves generator and discriminator parameters toward task-adapted weights, enabling further adaptation on a sampled class.

3. Few-shot Image Generation with Reptile

  • Few-shot image generation is defined over tasks that each correspond to one image class, with loss based on humans discriminating generated from real images.The paper does not conduct human benchmarking and leaves it for future work.Source: §3 · p. 2
  • Reptile meta-training adapts copied generator and discriminator weights on sampled task images, then updates the initial parameters toward those adapted weights.The inner loop takes K updates; the outer loop uses Adam on the difference between initial and adapted parameters.Source: §3 · p. 3
  • At generation time, copied meta-trained weights are adapted on a few images from a test class before the generator produces novel images.The adaptation loop updates the discriminator with Wasserstein GP loss and the generator with Wasserstein loss.Source: §3 · p. 3

4. Datasets

The experiments use MNIST and Omniglot character datasets, alongside FIGR-8, a larger collection of varied black-and-white icons. FIGR-8 includes many classes and is presented as a challenging benchmark for few-shot image generation.

4.1. MNIST

  • MNIST provides 60,000 training images: 28×28 grayscale examples from its 10 digit classes.The authors chose it because its simplicity allows rapid iteration on model ideas.Source: §4.1 · p. 3

4.2. Omniglot

  • Omniglot contains 1,623 character types from 50 alphabets, each handwritten once by 20 individuals.Its larger class count than MNIST supports testing out-of-sample performance across a wider set of image classes.Source: §4.2 · p. 3

4.3. FIGR-8

  • FIGR-8 contains 1,548,944 black-and-white icons across 18,409 classes, with at least eight images per class.The dataset includes varied samples and complex patterns, and is described as a tough yet achievable few-shot image-generation benchmark.Source: §4.3 · pp. 3–4

5. Experiments

FIGR is evaluated on previously unseen classes after fine-tuning on support images for 10 gradient steps. Results vary by dataset and image complexity, from good MNIST and simpler Omniglot samples to limited human-believability on more complex images.

5.1. Model architecture

  • The generator and discriminator use 18-layer residual networks with layer normalization; the model uses Wasserstein loss with gradient penalty, and PReLU activations.Generator layer normalization avoids batch-normalization running statistics, which are incompatible with Reptile’s meta-update.Source: §5.1 · p. 4

5.2. Empirical Validation

  • 50,000 meta-training steps produced good MNIST results on the held-out digit 9.Digits 0–8 were training classes; the figure shows results after 10 gradient steps.Source: Figure 3 · p. 5, §5.2 · p. 5
  • 140,000 meta-training steps yielded good results on simpler Omniglot characters, while most generations of more complex characters remained distinguishable from real ones after 230,000 steps.Figures 4 and 5 show the simpler and more complex character cases, respectively.Source: Figure 4 · p. 5, Figure 5 · p. 5, §5.2 · p. 5
  • 150,000 meta-training steps with n = 8 on 64×64 Omniglot images produced characters that could conceivably fool a human judge about half the time.The result followed fine-tuning for 10 gradient steps.Source: §5.2 · p. 5, Figure 6 · p. 6
  • 80,000 and 90,000 meta-training updates on FIGR-8 yielded quickly learned features such as birdlike shapes, though none of the shown generations could fool a human.The figures use n = 8 and 10 gradient steps.Source: Figure 7 · p. 6, Figure 8 · p. 6, §5.2 · p. 6

6. Conclusion

FIGR uses Reptile to train GANs for few-shot image generation, producing unseen-class images from as few as four samples on MNIST and Omniglot. The paper also introduces FIGR-8 and identifies directions for future work.

6. Conclusion

  • Reptile trains GANs for few-shot image generation: after meta-training on similar image classes, it generates unseen-class images from as few as 4 samples on MNIST and Omniglot.The approach uses no lengthy inference time, external memory, or additional data; preliminary FIGR-8 results show learning a complex concept such as “bird”.Source: §6 · p. 6
  • FIGR-8 contains over 18,409 classes and over 1,548,944 images, and the authors hope it will become a benchmark for few-shot image generation.Source: §6 · p. 6
  • Future work includes larger or multichannel images, meta-training through batch normalization, broader GAN architectures, and ImageNet-based pretraining for fine-tuning and transfer.Source: §6 · p. 6

Related papers

Loading 1901.02199v1…