PaperlayerView exact revision on arXiv

Source-linked AI summary

Only a Matter of Style: Age Transformation Using a Style-Based Regression Model

Yuval Alaluf, Or Patashnik, Daniel Cohen-Or

arXiv:2102.02754v2cs.CV

TL;DR

Age transformation must alter facial appearance substantially while preserving identity. The paper introduces SAM, which conditionally encodes real faces into a fixed StyleGAN using age-regression guidance and learns a non-linear latent path. It reports fine-grained age control, competitive evaluations, and further editing capabilities, while noting limitations from StyleGAN’s domain and age-predictor generalization.

  • Problem

    Age transformation must model changes in facial texture and head shape while preserving identity across potentially large age differences.

  • Method

    SAM encodes a real face and target age into StyleGAN’s latent space, using a pretrained age-regression network to guide the generated latent codes.

  • Results

    SAM provides fine-grained control through regression-based age progression and learns a more disentangled, non-linear latent path than methods using prior path assumptions.

  • Takeaways & Limitations

    The method supports additional fine-grained editing of aging results, including hair color and expression.

  • Takeaways & Limitations

    SAM is limited by StyleGAN’s domain and by the age predictor’s ability to generalize across age groups, particularly for children under five.

Abstract

from arXiv · show

The task of age transformation illustrates the change of an individual's appearance over time. Accurately modeling this complex transformation over an input facial image is extremely challenging as it requires making convincing, possibly large changes to facial features and head shape, while still preserving the input identity. In this work, we present an image-to-image translation method that learns to directly encode real facial images into the latent space of a pre-trained unconditional GAN (e.g., StyleGAN) subject to a given aging shift. We employ a pre-trained age regression network to explicitly guide the encoder in generating the latent codes corresponding to the desired age. In this formulation, our method approaches the continuous aging process as a regression task between the input age and desired target age, providing fine-grained control over the generated image. Moreover, unlike approaches that operate solely in the latent space using a prior on the path controlling age, our method learns a more disentangled, non-linear path. Finally, we demonstrate that the end-to-end nature of our approach, coupled with the rich semantic latent space of StyleGAN, allows for further editing of the generated images. Qualitative and quantitative evaluations show the advantages of our method compared to state-of-the-art approaches.

1 INTRODUCTION

Age transformation must make substantial changes to facial texture and head shape while preserving identity, especially across lifelong age gaps. SAM addresses this with conditional StyleGAN encoding, age-regression guidance, and a learned non-linear latent path.

  • Motivation: Age transformation requires changing facial texture and head shape while preserving identity, a challenge that grows with larger age shifts.The paper highlights lifelong changes such as transforming faces from age 5 to age 85.
  • Limitations of prior work: Existing image-to-image methods map faces between predefined age groups, but correlated groups hinder meaningful changes between ages and require costly age-annotated data.Large-scale collection of age-annotated data is described as tedious.
  • SAM: SAM encodes an input face and target age into StyleGAN style vectors, using a fixed generator and a pretrained age-regression network to guide the desired transformation.The method is trained with supervision from readily available pretrained networks rather than directly using labeled data.
  • SAM: Unlike latent-space methods that assume a linear age path, SAM learns a more precise non-linear path that is less entangled with other facial attributes.The learned path is analyzed as an end-to-end model of age transformation on real facial images.
  • Additional editing: SAM’s end-to-end design and StyleGAN latent space support additional fine-grained editing of generated aging results, including hair color and expression.These edits are presented as an additional capability of the generated images.
  • Results: Qualitative and quantitative evaluations report that the style-based regression method outperforms current state-of-the-art methods.The paper presents the method as a fine-grained model of the age transformation process.

2 RELATED WORK

Prior work approaches face aging through age-group translation or latent-space manipulation, but these strategies face difficulties with correlated age groups, real-image realism, and assumed latent paths. SAM directly encodes real faces conditioned on target age and learns a non-linear traversal without prior path assumptions.

  • Prior approaches: Face-aging methods commonly use image-to-image translation between predefined age groups or manipulation of a pretrained GAN’s latent space.These represent the two principal methodological directions described in the related work.
  • Image-to-image translation: Age-group translation methods struggle to disentangle age from correlated attributes and may require paired or age-labeled data that are tedious to collect.The related work describes age labels and translation between multiple age groups as common design choices.
  • Latent-space manipulation: Latent-space approaches often rely on traversing a presumed linear path controlling age, an assumption difficult to satisfy in a fully disentangled latent space.The paper also notes that these methods often struggle to produce realistic transformations on real images.
  • Latent-space manipulation: GAN inversion projects real images into a generator’s latent space by finding a latent representation that best approximates the input image.The paper situates inversion as a preliminary step in many latent-space editing methods.
  • SAM: SAM directly encodes a real face and desired target age into StyleGAN’s latent representation using an encoder paired with a fixed pretrained generator.The encoder produces style vectors that capture the desired transformation before image generation.
  • SAM: SAM learns a non-linear traversal path without prior assumptions, making it less sensitive to entanglement with other latent-space attributes.This distinguishes the method from approaches that impose a prior on the path used to control age.

3 METHOD

SAM models age transformation as conditional image-to-image translation: an encoder maps an input face and target age to StyleGAN style vectors, while training losses guide age accuracy, identity preservation, perceptual fidelity, and cycle consistency. The method uses a fixed pretrained generator and supports continuous target-age control in an unsupervised setting.

  • Architecture: SAM combines an aging encoder with a fixed pretrained StyleGAN generator to produce an output face at a specified target age.The encoder generates style vectors that the generator converts into the age-transformed image.
  • Unsupervised training: Cycle consistency addresses the absence of paired images by transforming an input and then recovering it at its source age.The recovered image is compared with the original input during training.
  • Training losses: SAM uses pixel and perceptual losses, with higher center-region weights to accommodate age-related changes in head shape.The objective also includes regularization, identity, age, and cycle losses.
  • Training losses: Identity preservation is encouraged with an age-dependent cosine-similarity loss whose weight decreases as the age shift increases.The minimum identity-loss weight is 0.5, reflecting that perceived identity can change across large age gaps.
  • Generated results: The method leverages StyleGAN's fixed generator to achieve high-quality aging results while retaining an end-to-end learned transformation.The figure caption attributes the image quality to the pretrained StyleGAN generator.

4 EXPERIMENTS

Experiments evaluate SAM against age-transformation and latent-space methods using identity, aging accuracy, image quality, and latent-path analyses. SAM generally preserves identity and improves aging and disentanglement, while showing limitations for very young targets and hair-related changes.

  • Comparison with Age Transformation Methods: SAM more naturally changes age-related facial features than LIFE and HRFAE, including facial hair, wrinkles, and jaw shape, while competing methods show artifacts or subtle texture changes.LIFE captures head-shape changes but introduces artifacts; HRFAE produces high-resolution images but limited age-related texture changes.
  • Comparison with Age Transformation Methods: All methods focus mainly on facial regions and struggle with hair-color changes and receding hairlines across aging.The authors note that preserving hair color may be undesirable because it can change over time, motivating additional editing techniques.
  • Comparison with Age Transformation Methods: SAM outperforms LIFE and HRFAE for most target ages, especially older ages above 60, but LIFE performs better for targets aged 5–10.Approximately 80% of the test set lies between ages 20 and 40, and the authors identify young-target performance as a SAM limitation.
  • Identity Evaluation: Human evaluations found the three methods similarly capable of preserving recognizable celebrity identity across target ages.Workers identified transformed celebrities from open-ended questions, with 150 responses per method and target age.
  • Comparison with Latent Space Methods: SAM’s end-to-end encoder learns the latent path without explicitly imposing a prior, supporting improved disentanglement compared with methods that operate directly on latent paths.The path analysis uses PCA projections of W+ age-transformed codes from real face images.
  • Comparison with Latent Space Methods: Compared with InterFaceGAN and StyleFlow, SAM provides better age disentanglement and visual quality on real facial images while preserving identity across ages.InterFaceGAN entangles age with glasses and hair color, whereas StyleFlow struggles with faithful transformation into the 60+ range.
  • Comparison with Latent Space Methods: SAM learns a nonlinear latent path whose behavior changes across age progression, unlike the strongly entangled linear path observed for InterFaceGAN.The authors also report smooth, disentangled age progression when traversing the learned manifold path from sampled StyleGAN latent codes.

5 DISCUSSION AND LIMITATIONS

The method has scope limitations tied to StyleGAN’s fixed representation, age-predictor generalization, and the difficulty of modeling complex age-related changes. It may also struggle with extreme inputs and very young children.

  • A fixed pre-trained StyleGAN simplifies training and produces high-quality images but constrains modeling of extreme poses, expressions, and accessories.
  • The method remains challenged by complex age-related changes such as receding hairlines and skin-color changes, despite editing techniques for global and local attributes.
  • The approach may struggle with extreme expressions or out-of-domain inputs absent from StyleGAN’s training data.
  • An age predictor must generalize across age groups, but biased adult-heavy datasets may reduce accuracy for children under age 5.

6 CONCLUSION

The paper presents an end-to-end method that maps a facial image and target age into a fixed StyleGAN latent space. It uses age-guided regression to learn a nonlinear age path, enabling fine-grained age control and subsequent editing.

  • The method maps a single input facial image and desired target age into the latent space of a fixed StyleGAN generator.
  • An age predictor guides latent-code generation, treating age progression as regression for finer control over the output age.
  • The approach learns a nonlinear latent path that better disentangles age from other facial attributes than prior path assumptions.
  • Improved disentanglement enables further fine-grained editing of generated aging results.

A IMPLEMENTATION DETAILS

The implementation uses pre-trained facial-image models and a fixed StyleGAN2 generator, with specified preprocessing, optimization, augmentation, and loss-weight settings.

  • Architectures: The encoder uses a pre-trained ResNet-IR backbone, while generation uses a fixed StyleGAN2 model trained on FFHQ.
  • Architectures: Identity embeddings come from ArcFace, and age prediction uses a classifier pre-trained on IMDB-WIKI.
  • Training: Training randomly sets the target age equal to the estimated source age with probability 0.33 to emphasize input reconstruction.
  • Training: The encoder is trained with Ranger at a constant learning rate of 0.001 using 256×256 inputs and horizontal-flip augmentation.
  • Losses: Generated 1024×1024 images are resized before loss computation, with separate preprocessing for identity and age networks.

B COMPARISON WITH MULTI-DOMAIN METHODS

The comparison evaluates SAM against FUNIT and StarGANv2, which use reference images for domain translation, while SAM specifies target ages directly. The reported qualitative comparisons highlight reference-image effects and SAM’s identity preservation.

  • Reference images substantially affect FUNIT and StarGANv2 outputs, altering attributes beyond age.
  • The comparison uses age groups spanning 3–6, 15–19, 30–39, 50–69, and 70–120.
  • FUNIT and StarGANv2 translate source images using reference images, whereas SAM sets the target age to each group’s median age.
  • Reference-image texture can produce unrealistic skin-tone changes and substantial variation across age groups, which is undesirable for continuous aging.
  • SAM more accurately preserves identity and key facial features than the SAMdirect variant while modeling lifelong age transformation.
  • The residual-based SAM variant compares favorably with SAMdirect, with lower values preferred in Table 3.

C ABLATION STUDY

The ablation study compares SAM training formulations and loss objectives, finding that residual latent-space learning improves identity preservation without harming aging accuracy, while the final loss configuration better captures some target-age and wrinkle changes.

  • Training Formulation: SAM learns a residual latent-space shift relative to the inverted input code rather than directly generating the age-transformed image.The encoder learns the difference between the original and age-transformed latent codes.
  • Training Formulation: Residual-based SAM improves identity preservation across target ages without reducing aging accuracy.The comparison evaluates SAM against SAM_direct, which directly predicts the transformed image.
  • Training Formulation: SAM better retains non-frontal poses, hairstyles, facial expressions, and eye-gaze direction in lifelong aging results.The visual comparison specifically highlights eye gaze and hairstyle preservation.
  • Loss Objectives: No single loss objective substantially changes the generated results, and all tested variants produce plausible outputs.The ablation removes individual objectives while keeping the remaining training configuration fixed.
  • Loss Objectives: The final loss configuration better captures younger target ages and facial-wrinkle changes, although removing forward-pass L2 and LPIPS losses can improve hair-color changes.The comparison uses target ages 5, 45, and 85.

D ADDITIONAL RESULTS

Additional results provide qualitative comparisons with competing age-transformation methods, multimodal style-mixing outputs, patch edits, and full-lifespan aging generated by SAM.

  • Additional Results: Because of dataset licensing restrictions, supplementary comparisons display pSp reconstructions instead of the original CelebA-HQ input images.This presentation choice applies to the additional CelebA-HQ visual results.
  • Additional Comparisons: Additional comparisons evaluate SAM against LIFE and HRFAE on CelebA-HQ test images.For LIFE comparisons, SAM uses the median age of each reference age group as the target age.
  • Additional Comparisons: SAM is also compared with InterFaceGAN and StyleFlow using inverted real face images.InterFaceGAN uses IDInvert with StyleGAN1, while StyleFlow uses pSp inversion into StyleGAN2.
  • Additional Results: Style-mixing SAM outputs with five reference images on layers 8–9 produces multiple age-transformation results per input.The procedure demonstrates multimodal generation from age-transformed outputs.
  • Additional Results: The additional figures include SAM patch-editing results, age-transformation results, and full-lifespan aging sequences.The supplementary visualizations cover patch edits, general transformations, and lifespan-wide aging.
Loading 2102.02754v2…