Source-linked AI summary
Few-Shot Adaptation of Generative Adversarial Networks
Esther Robb, Wen-Sheng Chu, Abhishek Kumar, Jia-Bin Huang
TL;DR
GANs typically need many training samples for high-quality synthesis, while few-shot settings pose a challenge for adaptation. FSGAN adapts pretrained GANs by tuning singular values while keeping singular vectors fixed. The paper reports favorable results against existing data-efficient GAN adaptation methods and highlights that few-shot FID can favor overfitting.
Problem
GANs typically require large target-domain datasets for high-quality synthesis, and scarce-data training can lead to memorization or instability.
Source: §1 · p. 1Method
FSGAN applies SVD to pretrained GAN weights and adapts their singular values while keeping the corresponding singular vectors fixed.
Source: §1 · p. 1, §3.1 · p. 3, §3.2 · p. 4Results
FSGAN shows favorable results over existing data-efficient GAN adaptation methods across close- and far-domain experiments and varied n-shot settings.
Source: §5 · p. 8Takeaways & Limitations
The paper concludes that FSGAN can adapt pretrained GANs with scarce target-domain images while preserving the ability to generate diverse and realistic samples.
Source: §5 · p. 8In few-shot synthesis, FID can favor overfit models because memorizing the sampled training set can approximate test-set statistics.
Source: §3.4 · p. 5
Abstract
from arXiv · showhide
Generative Adversarial Networks (GANs) have shown remarkable performance in image synthesis tasks, but typically require a large number of training samples to achieve high-quality synthesis. This paper proposes a simple and effective method, Few-Shot GAN (FSGAN), for adapting GANs in few-shot settings (less than 100 images). FSGAN repurposes component analysis techniques and learns to adapt the singular values of the pre-trained weights while freezing the corresponding singular vectors. This provides a highly expressive parameter space for adaptation while constraining changes to the pretrained weights. We validate our method in a challenging few-shot setting of 5-100 images in the target domain. We show that our method has significant visual quality gains compared with existing GAN adaptation methods. We report qualitative and quantitative results showing the effectiveness of our method. We additionally highlight a problem for few-shot synthesis in the standard quantitative metric used by data-efficient image synthesis works. Code and additional results are available at http://e-271.github.io/few-shot-gan.
1 INTRODUCTION
GANs need large, diverse target-domain datasets for high-quality synthesis, while scarce data can lead to memorization, instability, or poor image quality. FSGAN adapts pretrained GANs by optimizing singular values with singular vectors fixed, and the paper reports higher image quality than several existing adaptation approaches.
1 INTRODUCTION
- With fewer than 1,000 training samples, GANs frequently memorize or become unstable, reducing diversity or visual quality.GAN training also faces challenging adversarial optimization and inadequate generation of rare modes.Source: §1 · p. 1
- Few-shot GAN adaptation is difficult: fine-tuning all weights can perform poorly, while constrained alternatives have reported blurry images or require compute- and memory-intensive architectures.The cited alternatives include batch-normalization adaptation and invertible flow-based models.Source: §1 · p. 1
- Figure 1 shows diverse toddler images generated by adapting FFHQ-pretrained StyleGAN2 with only 30 CelebA images, alongside smooth interpolation between random seeds.The caption presents the interpolation as evidence that samples are novel rather than simply memorized training images.Source: Figure 1 · p. 2
- FSGAN reports higher image quality after adaptation than methods that fine-tune all weights, individual layers, or batch-normalization statistics.The method applies SVD to pretrained generator and discriminator weights, then adapts their singular values while fixing the left and right singular vectors.Source: §1 · p. 2
2 BACKGROUND
GANs generate images by mapping random noise to a learned image distribution through adversarial training, and high-quality synthesis has generally relied on large datasets. Prior sample-efficient methods differ in which parameters they adapt; FSGAN instead tunes singular values in pretrained weight space.
2 BACKGROUND
- GANs learn to map random noise to an image distribution through adversarial training, and typically produce sharper images than pixel-reconstruction methods.High-quality generation has relied on large datasets of high-quality images, and GANs may omit outlier modes even with large training sets.Source: §2 · p. 2
- Sample-efficient synthesis methods commonly use pretraining or simultaneous training, with adaptation ranging from all network weights to discriminator layers or batch statistics.Flow-based methods show promise for few-shot adaptation but require compute- and memory-intensive architectures and data-dimensional latent spaces.Source: §2 · p. 2
- For each convolutional layer, FSGAN uses 16K learnable parameters versus SSGAN’s 23K; FSGAN adapts singular values in both convolutional and fully connected layers, while SSGAN adapts only convolutional layers.Source: §2 · p. 3, Figure 2
- Unlike GANSpace’s latent-space directions for editing one image, FSGAN performs weight-space SVD for domain adaptation; unlike one-shot resynthesis, it transfers knowledge from a pretrained GAN.The paper describes one-shot resynthesis as modeling the internal feature distribution of a single image without pretraining.Source: §2 · p. 3
3 FEW-SHOT GAN
FSGAN identifies semantically meaningful directions in pretrained weight space by decomposing GAN weights and adapting singular values while fixing singular vectors. The method specifies how it processes layers and trains, and it argues that few-shot FID can favor overfitting rather than reliable synthesis quality.
3.1 OVERVIEW
- Varying leading singular values changes meaningful image attributes: style layers affect age, skin tone, and pose, while synthesis layers alter face dimensions and finer appearance.The overview interprets changes in singular values as changing entanglement between orthogonal factors of variation in the data.Source: §3.1 · p. 3, Figure 3 · p. 4
3.2 ADAPTATION PROCEDURE
- FSGAN decomposes each generator and discriminator layer, then freezes pretrained singular vectors and optimizes singular values under a GAN objective to match target-domain outputs.For convolutional weights, it flattens spatial and input-feature dimensions before SVD, aiming to preserve the pretrained spatial-feature relationship.Source: §3.2 · p. 4
3.3 TRAINING & INFERENCE
- Training is capped at 20K images, or 16K for 5-shot, because longer training can cause overfitting or quality degradation.At inference, truncation with ψ = 0.8 is less restrictive and enables more diversity in generated images.Source: §3.3 · p. 5, Figure 4 · p. 5
3.4 EVALUATION IN FEW-SHOT SYNTHESIS
- In few-shot synthesis, held-out FID can still favor overfit or poor-quality models because memorizing the sampled training set can approximate test-set statistics.The authors recommend supplementing FID with additional metrics and extensive qualitative results; for face transfer, they also report sharpness and face quality.Source: §3.4 · p. 5
4 EXPERIMENTS
The experiments evaluate FSGAN on close- and far-domain adaptation and across different target-set sizes. They report qualitative advantages over existing methods, including robust high-quality adaptation across tested n-shot settings.
4.1 SETTINGS
- The evaluation covers 5–100 target images across CelebA, Portraits, Anime Rem, and Van Gogh landscapes, comparing FSGAN with TGAN, FreezeD, and SSGAN.The experiments also report FID, face quality, and sharpness; the supplied Table 1 caption gives metric scope but no values.Source: §4.1 · pp. 5–6, Table 1 · p. 6
4.2 NEAR-DOMAIN ADAPTATION
- In close-domain FFHQ-to-single-ID CelebA adaptation, FSGAN generates more natural faces, while FID correlates poorly with qualitative evaluation.Face quality and sharpness metrics show competitive performance across adaptation settings.Source: Figure 5 · p. 6, §4.2 · p. 7
4.3 FAR-DOMAIN ADAPTATION
- Across three far-domain transfers, FSGAN makes changes to match target semantics, proportions, and textures while maintaining image quality.The settings include Churches-to-Van Gogh paintings, FFHQ-to-Art portraits, and FFHQ-to-Anime Rem ID.Source: §4.3 · pp. 7–8
4.4 N-SHOT SETTINGS
- FSGAN transfers semantic features while preserving quality across all tested n-shot settings, whereas FreezeD’s limited-step and longer-run variants trade off adaptation quality and degradation.The experiments note variance across n-shot settings for all methods as the data distribution changes.Source: §4.4 · p. 8
5 CONCLUSIONS
The conclusion presents Few-Shot GAN as a method for adapting pretrained GANs when target images are scarce. Its experiments cover close- and far-domain transfers and multiple n-shot settings, with favorable results against existing data-efficient adaptation methods.
5 CONCLUSIONS
- Few-Shot GAN adapts pretrained GANs to new target domains when training images are scarce.The paper presents the method as simple and effective.Source: §5 · p. 8
- The method uses SVD of pretrained convolutional and fully connected layer weights to identify a semantically meaningful parameter space for adaptation.The conclusion describes this as the method’s core idea.Source: §5 · p. 8
- Across close- and far-domain experiments and various n-shot settings, the method preserves diverse, realistic sample generation while showing favorable results against existing data-efficient GAN adaptation methods.Source: §5 · p. 8