Source-linked AI summary
Fader Networks: Manipulating Images by Sliding Attributes
Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, Marc'Aurelio Ranzato
TL;DR
The paper addresses unsupervised image manipulation when desired attribute transformations are ill-defined. It introduces Fader Networks, which enforce attribute-invariant latent representations while decoding with user-controlled values. The model produces natural, identity-preserving edits, supports continuous and multiple-attribute control, and outperforms a prior output-adversarial baseline in reported evaluations.
Problem
Unsupervised attribute manipulation lacks examples of the desired transformed image, although realistic edits must preserve the original image's salient content.
Method
Fader Networks use an encoder-decoder with an attribute-invariant latent space, reconstructing images from latent representations and user-selected attribute values.
Results
The model significantly changes perceived attributes while preserving naturalness and identity, and outperforms prior output-adversarial methods in reconstruction loss and human-rated generation quality.
Takeaways & Limitations
Continuous latent-space attribute control provides a simpler training pipeline that is readily extended to multiple attributes without applying a GAN to the decoder output.
Takeaways & Limitations
Identity preservation was not separately evaluated because the model's reconstruction quality was very good and the authors did not observe identity problems.
Abstract
from arXiv · showhide
This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space. As a result, after training, our model can generate different realistic versions of an input image by varying the attribute values. By using continuous attribute values, we can choose how much a specific attribute is perceivable in the generated image. This property could allow for applications where users can modify an image using sliding knobs, like faders on a mixing console, to change the facial expression of a portrait, or to update the color of some objects. Compared to the state-of-the-art which mostly relies on training adversarial networks in pixel space by altering attribute values at train time, our approach results in much simpler training schemes and nicely scales to multiple attributes. We present evidence that our model can significantly change the perceived value of the attributes while preserving the naturalness of images.
1 Introduction
The paper targets unsupervised manipulation of natural images by changing annotated attributes while preserving the underlying person or image content. Fader Networks disentangle attributes in a latent space, enabling continuous control and realistic multi-attribute edits.
- Unsupervised attribute editing seeks realistic versions of the same image with changed properties such as gender, age, or expression.
- Fader Networks encode an image into a latent representation and decode it with user-selected attribute values.
- Continuous attribute values at inference time control how strongly each attribute appears in the generated image.
- The latent representation is trained to be attribute-invariant while retaining enough information for reconstruction, so one representation can produce multiple attribute versions.
- The approach uses latent-space adversarial training rather than pixel-space adversarial training, yielding simpler pipelines and easier extension to multiple attributes.
- Experiments on CelebA indicate realistic attribute edits that preserve natural appearance and identity, while outperforming prior output-adversarial methods in reconstruction loss and human-rated generation quality.
2 Related work
Prior work spans fully supervised transformation learning, fully unsupervised factor discovery, and intermediate settings with specified attributes but no target transformations. The paper positions its approach between these extremes and contrasts latent-space invariance with pixel-space adversarial translation methods.
- Fully supervised methods learn specified transformations from input, transformation, and result examples.
- Fully unsupervised methods disentangle unspecified factors of variation, but may struggle to discover high-level concepts such as gender or age.
- This work occupies an intermediate setting with specified attributes but without supervision for the intended modified result.
- Related domain-transfer methods typically use pixel-space adversarial training and separate components per domain, which does not scale well to multiple attributes.
- The latent-invariance criterion resembles adversarial training used in domain adaptation, fair classification, and robust inference, but here it forces the decoder to use attribute information for generation.
- A feature-interpolation alternative supports attribute-based face alterations but has costly inference and depends on attributes available during pre-training.
3 Fader Networks
Fader Networks learn attribute-invariant latent representations so a decoder can reconstruct images while responding to user-selected attribute values. The approach uses adversarial training in latent space to preserve image content and enable attribute manipulation.
- Problem and goal: The model learns from image-attribute pairs to generate versions of an input image matching any target attribute vector.Attributes are binary in the described setting, with extension to categorical attributes suggested.
- Architecture: An encoder maps an image to a latent representation, while a decoder reconstructs the image from that representation and its attributes.The encoder is convolutional and the decoder is deconvolutional.
- Motivation: Without additional constraints, the decoder can ignore attributes, making attribute changes at test time ineffective.This motivates constraining the latent representation to be invariant to the attributes of interest.
- Adversarial training: Adversarial latent-space training makes a discriminator predict attributes from the latent code while training the encoder-decoder to prevent that prediction.The encoder-decoder jointly optimizes reconstruction and attribute invariance.
- Training objective: The reconstruction–invariance trade-off is controlled by λE: large values can blur images, whereas low values weaken attribute effects.The coefficient balances reconstruction quality against invariance of the latent representations.
- Inference: At inference, users can vary attribute values, including continuously, to control how strongly attributes appear in generated images.The discriminator is discarded at test time, and the decoder receives the selected attributes.
4 Implementation
The implementation uses convolutional encoder-decoder components, explicitly injects attributes throughout decoding, and evaluates models using perceptual naturalness and attribute-swap accuracy.
- Network architecture: The encoder uses seven convolutional layers with 4 × 4 kernels, stride 2, padding 1, leaky-ReLUs, and batch normalization.Each encoder layer halves the spatial dimensions.
- Network architecture: For 256 × 256 inputs, the latent representation contains 512 feature maps of size 2 × 2.Six layers produced similar results, while eight layers significantly reduced performance.
- Attribute conditioning: The decoder receives attribute information by appending the attribute code as constant input channels to every convolutional layer.With n binary attributes, the concatenated one-hot code has size 2n.
- Evaluation and selection: Naturalness is the percentage of images judged real by humans, while swap accuracy is human classification accuracy for attribute values.Table 1 reports both perceptual measures for each model.
- Discriminator: The discriminator consists of a C512 layer followed by fully connected layers of sizes 512 and n.The final layer predicts the n attributes.
- Training schedule: Discriminator-loss scheduling starts λE at 0 and increases it linearly to 0.0001 over 500,000 iterations.The authors report that this scheduling was critical in their experiments.
- Evaluation and selection: Model selection uses reconstruction MSE and attribute-swap accuracy measured by a classifier on decoded validation images.The two criteria assess reconstruction quality and whether swapped attributes are expressed.
5 Experiments
Experiments on CelebA and Oxford-102 evaluate Fader Networks for realistic attribute swapping, including multiple facial attributes and continuous flower-color control. On CelebA, the model produces high-quality images, strong attribute-swap accuracy, and generally higher naturalness than IcGAN, though naturalness varies by attribute.
- Experimental setup: The CelebA experiments use 200,000 celebrity images annotated with 40 attributes, with IcGAN trained on the same dataset as the baseline.Images were cropped to 178 × 178 and resized to 256 × 256; all paper figures and evaluations use test-set images.
- Qualitative evaluation: Fader Networks generate high-quality face images while changing attributes such as glasses, with plausible attribute-specific details that vary according to the input face.The results suggest the model preserves information needed to generate different versions of a face rather than adding a generic modification.
- Evaluation caveat: Identity preservation was not separately evaluated because the authors report very good reconstruction quality, with RMSE 0.0009 versus 0.028 for IcGAN.The paper identifies identity assessment as a technical consideration but says it did not observe the issue in its reconstructions.
- Quantitative evaluation: Fader Networks achieve 88.4%, 75.2%, and 78.8% naturalness accuracy for reconstructions, while IcGAN reconstruction or swap accuracy does not exceed 23%.For Fader Networks swaps, naturalness ranges from 79.0% for mouth opening to 31.4% for smile.
- Quantitative evaluation: FADNET SWAP reaches 66.2% mouth, 76.6% glasses, and 97.1% smile accuracy, compared with IcGAN's 10.1%, 47.5%, and 9.9% on the same attributes.The classification experiments measure whether human workers perceive the intended swapped attribute.
- Multi-attribute swapping: The model can jointly swap gender, opened eyes, and eye glasses while generating convincing images in this more difficult multi-attribute setting.Figure 4 presents these swaps using the same model, with original images shown on the left.
- Flowers dataset: On Oxford-102, reconstructed flowers are evaluated using color attributes derived from captions, and changing the pink value shifts flowers toward pink, yellow, or orange.Each flower has ten captions; a color label is assigned when it appears in at least five captions.
6 Conclusion
The paper proposes generating image variations by changing attribute values while enforcing attribute invariance in the latent space. It reports realistic high-resolution images without applying a GAN to decoder outputs and notes possible extension beyond vision.
- Conclusion: Fader Networks generate image variations by changing attribute values while enforcing invariance of the latent space to those attributes.This is the central approach stated in the conclusion.
- Conclusion: The method generates realistic high-resolution images without requiring a GAN on the decoder output.The authors present this as a key advantage over many recent models.
- Conclusion: The approach could extend to domains such as speech or text, where backpropagation through decoder generation can be challenging.The conclusion notes that visual-quality methods such as PatchGAN could also be applied on top of the model.