Improved Input Reprogramming for GAN Conditioning

1 Introduction

• Conditional GANs have attracted attention for generating class-conditional data and support applications including image generation, text-to-image generation, image translation, and text-to-speech.GANs also model complex high-dimensional distributions, including images, videos, audio, and text.Source: §1 · p. 1
• GAN conditioning uses a pretrained unconditional generator and labeled data to learn a conditional generator, assuming the labeled classes have separable class-conditional distributions.The formulation also describes the labeled dataset as class-conditional samples paired with labels.Source: §1 · p. 1
• Pipelined data collection, unlabeled-then-labeled streams, and publicly available models trained on private data motivate reusing an unconditional generator when labeled data arrive.In the streaming scenario, the unlabeled data may need to be discarded because of storage constraints.Source: §1 · p. 2
• Existing conditioning options are training a CGAN from scratch, fine-tuning a UGAN, or input reprogramming, whose prior versions handle only one class and require repetition for multi-class data.Scratch training can degrade with scarce or noisy labels, while fine-tuning can suffer catastrophic forgetting; comparisons across label-data conditions remained unclear.Source: §1 · p. 2
• INREP+ uses an invertible modifier to prevent latent-space class overlap, shared networks to reduce memory demands, and a PU-learning discriminator loss to improve training stability.The modifier transforms noise into class-conditional noise, which the fixed unconditional generator maps to a class-conditional sample.Source: §1 · p. 2
• With labeled data as small as 1% or 10% of the unlabeled data used for UGAN training, INREP+ outperforms other approaches across datasets and is robust to noisy, imbalanced labels.The study also reports state-of-the-art performance across various quantitative measures.Source: §1 · p. 2

2.1 Learning CGAN without using UGAN

• From-scratch CGAN training can degrade with scarce labels and require substantial resources, while ACGAN and projection-based conditioning have identified failure cases.The analyzed ACGAN failures include suboptimal local minima on separable and non-separable data; projection-based conditioning can deviate from exact conditional distributions with unsuitable class embeddings.Source: §2.1 · pp. 3–4

2.2 Fine-tuning UGAN into CGAN

• Fine-tuning can reduce training time and labeled-data needs while remaining competitive, but architecture adaptation and catastrophic forgetting are concerns.Catastrophic forgetting may affect the quality of samples from a well-trained unconditional generator.Source: §2.2 · p. 4

2.3 Input reprogramming

• Input reprogramming keeps the pretrained UGAN intact and adds a controller that learns class-conditional latent vectors, offering computing and memory savings.The existing GAN-REPROGRAM approach focuses on one condition at a time, creating memory demands as the number of classes grows, and underperforms recent CGAN algorithms on complex datasets.Source: §2.3 · pp. 4–5
• INREP+ improves input reprogramming with invertible networks, a Positive-Unlabeled-based loss, and shared class-network weights, and examines noisy and imbalanced supervision.The study reports sharper distributions with small amounts of supervision and consistent performance improvements.Source: §2.3 · p. 5

2.4 Summary of comparison

• With scarce labels, INREP+ performs best among methods reusing a UGAN; input reprogramming is the best solution when both labeled data and resources are restricted.With sufficient labeled data and computing resources, training a CGAN without the UGAN can achieve the best performance; fine-tuning is a candidate when data is plentiful but resources are restricted.Source: §2.4 · p. 5, Table 1 · p. 5

3.1 Revisiting input reprogramming for GAN conditioning

• Input reprogramming conditions a pretrained unconditional generator by transforming its noise input while leaving the generator unchanged.A modifier maps noise and a class label to conditional noise, which the frozen generator uses to produce class-conditional samples.Source: §3.1 · pp. 5–6
• GAN-REPROGRAM can produce overlapping class-specific latent regions, wrongly reject desired samples as fake during training, and require vast memory as classes grow.The false-rejection dynamic can make reprogramming difficult, especially with low supervision.Source: §3.1 · p. 6

3.2 INREP+: An improved input reprogramming algorithm

• INREP+ uses one conditional modifier and a shared discriminator base with class-specific linear heads, reducing the parameters and weights needed to store.The design supports memory scalability across classes while retaining separate conditional discriminator outputs.Source: §3.2 · p. 6
• An invertible modifier maps different random noises to distinct latent vectors, preventing class-conditional latent regions from overlapping.The mapping has a dimension gap, du < dz = dzy; the authors state this does not significantly affect its ability to capture the conditional latent distribution.Source: §3.2 · pp. 6–7
• INREP+ treats generated data as unlabeled in a PU-learning view and uses a revised discriminator loss to address false rejection of in-class generated samples.For real data, its loss more strongly encourages predictions near 1 for positive samples than the standard negative log-loss.Source: §3.2 · pp. 6–7

3.3 INREP+’s optimality under ideal setting

• At global optimum with the ideal unconditional generator and discriminator fixed, INREP+ generates the true conditional distribution.The theorem states this condition holds if and only if the generated distribution equals the true conditional distribution.Source: §3.3 · p. 8

4.1 Performing GAN conditioning with INREP+ on various data

• INREP+ produces mode-covering Gaussian-mixture samples, correctly labeled MNIST and face samples, and CIFAR10 samples that are generally high-quality and correctly conditioned.The face experiment uses an FFHQ-pretrained StyleGAN and CelebA labels for wearing glasses and male.Source: §4.1 · p. 9, Figure 5 · p. 9, Figure 6 · p. 10

4.2 GAN conditioning with different levels of supervision

• At 1% labels, INREP+ has Intra-FID 76.24 on CIFAR10 and 238.24 on CIFAR100, with gaps of 38.27 and 11.53 to the second-best models.It also leads in recall and CAS on CIFAR10 at 1%, 10%, and 20% labeled data.Source: §4.2 · p. 10, Table 2 · p. 11, Table 3 · p. 11
• From 20% labels onward, CGANs outperform INREP+, though its full-supervision CIFAR10 gaps to ProjGAN are only 1.1 FID and 7.5 Intra-FID.Table 2 compares CIFAR10 and CIFAR100 across label fractions and reports small FID gaps for INREP+ at higher supervision.Source: §4.2 · p. 10, Table 2 · p. 11

4.3 Robustness to class-imbalanced data and noisy supervision

• On imbalanced CIFAR10, INREP+ obtains overall FID 13.26 and minor-class Intra-FIDs 69.48 and 61.68, near the major-class average of 60.06.Classes 0 and 1 use 10% of their labeled data; other classes remain unchanged.Source: §4.3 · p. 11, Table 4 · p. 12
• With label-flip probability 0.4, INREP+ has an FID increase of 0.16 and Intra-FID 63.15; input-reprogramming methods show smaller increases than other methods.The reported noise setting flips bird→airplane, cat→dog, deer→horse, and truck→automobile; label noise mainly affects corresponding classes for INREP+ and GAN-REPROGRAM.Source: §4.3 · pp. 11–12, Table 5 · p. 12

4.4 Ablation study

• Ablations show both invertibility and PU-based loss improve CIFAR10 Intra-FID, with invertibility having the larger measured effect.The reported largest decreases are 15.1 for invertibility and 6.3 for PU-based loss.Source: §4.4 · p. 12

Ablation study

• On CIFAR10, both components improve Intra-FID, but invertibility has the stronger measured effect than PU-based loss.The table reports reductions of nearly 11.7 and 15.1 for invertibility, versus approximately 2.9 and 6.3 for PU-based loss.Source: Table 6 · p. 12

Limitations

• INREP+ depends greatly on how well the pretrained unconditional generator learns the data distribution and organizes its latent space.The authors note capacity gaps between unconditional and conditional GANs on large-scale datasets, leaving improvement there challenging.Source: §5 · pp. 12–13

Beyond GAN: Can we apply INREP+ to other generative models?

• The framework could replace its unconditional GAN with a Glow or VAE generator; the authors note VAE-pretrained generators may help stabilize GAN training and reduce mode collapse.This is presented as a possible application beyond the paper's GAN focus.Source: §5 · p. 13

Enhancing input reprogramming to prompt tuning

• INREP+ may also offer a way to learn prompts for generative tasks while freezing the generative model and saving computation.The discussion connects this possibility to prompt tuning and prefix tuning.Source: §5 · p. 13

6 Conclusion

• The paper defines GAN conditioning and reviews three approaches, focusing on input reprogramming as the best-performing approach when labeled data is scarce.Source: §6 · p. 13
• INREP+ addresses issues in prior input reprogramming with an invertible modifier, a multi-head discriminator, and a Positive-Unlabeled-based GAN loss.Source: §6 · p. 13
• With perfect unconditional generators, ideal INREP+ training provably learns true conditional distributions; empirically, it outperforms alternatives with scarce labels and is more robust to label noise and class imbalance.Source: §6 · p. 13

A.1.1 Examples of the failure of ACGAN

• With a perfect discriminator, 1-D data x~N(y,1), y=2 Bern(1/2)-1, and generator G(z,y)~N(y,v²) parameterized by standard deviation v∈R+, W-ACGAN’s optimal v*(λ) tends to 0 as λ→∞.Source: §A.1.1 · p. 17
• With a perfect discriminator, when the classification term dominates LS, gradient-based training can learn ℓ* ≠ 1 at a bad critical point; the example uses ℓ* = −0.6, α* ≈ 1.26π, and λ* = 2.Source: §A.1.1 · pp. 18–19

A.1.2 Examples of the failure of ProjGAN

• ProjGAN’s projection operation reduces to expectation matching in a simple setting, where the authors show it can mislearn exact conditional distributions.Even after the generator matches the true conditionals, discriminator embedding vectors can encourage it to deviate.Source: §A.1.2 · p. 19
• After exact conditional matching, repeated ProjGAN updates cause the generator to diverge unless v0,t* + v1,t* = 0.The analysis attributes the continued update to a nonvanishing generator gradient.Source: §A.1.2 · p. 21

• For a perfect continuous generator, the paper constructs condition-specific noise z_y such that G(z_y) has distribution x|y=y.The result assumes continuous densities, a locally Lipschitz inverse around each preimage, a finite Jacobian operator norm, and continuity of p(y|x) in x.Source: §A.2 · p. 21
• The construction works by selecting noise-space regions associated with the target class and bounding their probability around each preimage.The proof uses the local Lipschitz property of G^-1 and the finite operator norm of the Jacobian.Source: §A.2 · pp. 21–22
• The construction’s probability bounds yield G(z_y) distributed as x conditioned on y.The proof normalizes the bounds and takes the radius to zero before concluding the distributional equality.Source: §A.2 · p. 22

• The optimal discriminator balances the true conditional density against the generated fake density using the PU class prior π_y.Its stated form is (1+π_y)p_data(x|y) divided by (1+π_y)p_data(x|y)+(1−π_y)p_gf(x).Source: §A.3 · p. 22
• The global modifier optimum is attained if and only if p_gf(x)=p_g(x)=p_data(x|y).The theorem assumes the ideal unconditional generator and discriminator are fixed.Source: §A.3 · p. 23
• Optimizing the modifier under the derived objective yields p_gf(x)=p_data(x|y), making the generated distribution match the target conditional.The proof expresses the objective using KL divergence.Source: §A.3 · p. 23

B.1 Datasets

• The study covers MNIST (28 × 28; 50,000 train/10,000 test), CIFAR10 (32 × 32; 50,000/10,000), CIFAR100 (32 × 32; 500/100 per class), CelebA [56] (>200,000 images), and FFHQ [55] (70,000 images at 1024 × 1024).Source: §B.1 · p. 23

B.2 Evaluation metrics

• FID measures quality and diversity using feature-space Gaussian Wasserstein-2 distance, with lower scores indicating better GAN performance.The stated metric is unsuitable for non-ImageNet-like images such as MNIST or Fashion-MNIST.Source: §B.2 · p. 23
• Intra-FID averages class-specific FID scores to measure class-conditioning performance.Source: §B.2 · p. 24
• CAS evaluates a classifier trained on generated conditional data by measuring its accuracy on real data, while recall measures coverage of the reference distribution.Higher recall indicates better coverage; CAS should approach the accuracy of a classifier trained on real samples when generated data match the real distribution.Source: §B.2 · p. 24

B.3 Architectures, hyperparameters, and implementation

• The evaluation targets reliable comparisons of conditioning methods, applying the same configuration to all models rather than optimizing synthesis performance.The authors note that state-of-the-art synthesis typically requires more computation and additional tricks.Source: §B.3 · p. 24
• For Gaussian mixtures, three fully connected blocks use non-saturating GAN loss and train for 10,000 steps with batch size 64.Training uses Adam with β1 = 0.5, β2 = 0.999, and learning rate 2 · 10^-4 for all networks.Source: §B.3 · p. 24
• For MNIST, DCGAN architectures use WGAN-GP, five critic steps, and 50,000 training steps with batch size 64.Adam learning rates are 2 · 10^-4 for the modifier and 2 · 10^-5 for the discriminator.Source: §B.3 · p. 24
• For CIFAR10 and CIFAR100, all models use ResGAN with spectral normalization and Hinge loss [81]; InRep+ uses i-ResNet modifiers with three layers.The InRep+ modifier uses five layers for FFHQ; the CIFAR architecture choice follows reported comparable DCGAN and ResGAN performance [62], with recommended configurations [62].Source: §B.3 · p. 24