PaperlayerView exact revision on arXiv

Source-linked AI summary

ALADDIN: Adversarial Latent Diffusion Distillation for One-Step Identity Swapping

Ivan Shchekotov, Timofei Gritsaev, Viacheslav Pirogov, Dmitry Vetrov

arXiv:2610.37503v1cs.CV

TL;DR

Face swapping aims to transfer identity while preserving target attributes, but diffusion methods are costly at inference and GAN methods can struggle to learn realism. ALADDIN initializes a one-step generator from a pretrained flow-matching model and trains it adversarially with identity and preservation constraints. It matches the strongest multi-step baseline with 28 times fewer network evaluations and an order of magnitude fewer training pairs.

  • Problem

    Diffusion-based face swapping offers realism but requires many network evaluations, while GAN methods trained from scratch must learn realism from adversarial signals alone.

    Source: §1 · p. 2, §2 · p. 2

  • Method

    ALADDIN initializes a one-step generator from a pretrained flow-matching model and trains it adversarially using diffusion features, identity supervision, and target-preservation constraints.

    Source: §1 · p. 2, §3 · p. 3, §3.1 · p. 3

  • Results

    ALADDIN matches APPLE in identity transfer, FID, and expression, improves on it in pose, and uses one network evaluation instead of 28 and an order of magnitude less training data.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The pretrained flow model supplies the realism prior both through generator initialization and as the discriminator’s feature extractor, without requiring a distillation objective.

    Source: §5 · p. 8

    Identity transfer degrades for unusual source identities and large age gaps; the weakest-target-preservation model also loses tattoos.

    Source: §5 · p. 8

Abstract

from arXiv · show

Face swapping transfers a source identity onto a target image while preserving the target's pose, expression, hair, and lighting. The most realistic current methods are diffusion-based, but they need many network evaluations per image. GAN-based methods are fast, yet trained from scratch they must learn realism from the adversarial signal alone and produce visibly weaker results. We present ALADDIN (Adversarial Latent Diffusion Distillation for One-Step Identity Swapping), a one-step face swapper obtained by initialising the generator from a pre-trained multi-step flow-matching model and training it adversarially under identity and target-preservation constraints. We expose a controllable trade-off between identity transfer and target-attribute preservation by training a student model with triplet pseudo-label supervision. We additionally identify a bias in the standard evaluation protocol, which favours methods trained against the same identity encoder used to score them, and re-evaluate all baselines with an independent one. ALADDIN is state-of-the-art among one-step methods in realism and target preservation at comparable identity transfer, and matches the best multi-step model with 28 times fewer network evaluations and an order of magnitude fewer training pairs.

1. Introduction

Face swapping seeks to transfer source identity while retaining target attributes, but diffusion-based realism requires many evaluations and GAN-based methods often produce less realistic results. ALADDIN combines a pretrained diffusion generator with adversarial training for one-step swapping and reports strong results alongside an evaluation-protocol correction.

1. Introduction

  • Face swapping transfers source identity while preserving target attributes such as background, hair, pose, expression, lighting, and accessories.The paper notes applications including film production and privacy protection [50].Source: §1 · p. 1
  • Diffusion-based face swapping offers strong realism but requires tens of network evaluations per image, whereas GAN inference takes one.Diffusion distillation seeks to retain generation quality while reducing inference cost.Source: §1 · p. 2
  • ALADDIN achieves state-of-the-art quality among one-step models, matches APPLE, and uses 28× fewer network evaluations and an order of magnitude fewer training pairs.The authors also identify positive bias in identity similarity estimates when methods are evaluated with the identity model used for training, and re-evaluate baselines independently.Source: §1 · p. 2
  • ALADDIN initializes its generator from a pretrained diffusion model and trains adversarially with a discriminator that reads diffusion features.The design combines diffusion realism with one-step GAN inference.Source: §1 · p. 2

2. Related work

Prior face-swapping approaches use GANs, pretrained StyleGAN inversion, or diffusion models, each with distinct trade-offs in realism, control, and inference cost. Recent one-step methods and multi-step systems motivate ALADDIN’s use of a non-distilled pretrained model and triplet supervision.

2. Related work

  • Early neural face-swapping GANs divide broadly into direct-generator and GAN-inversion approaches; direct methods combine source identity with target attributes, as in SimSwap, InfoSwap, HiFiFace [49], and FaceDancer.Source: §2 · p. 2
  • Trained from scratch, direct GAN methods learn realism from adversarial signals competing with identity and attribute objectives, and can artifact under large pose, shape, or occlusion changes.StyleGAN inversion methods inherit the generator’s realism but may lose identity detail or leave its well-behaved latent region during embedding [38] [47].Source: §2 · p. 2
  • Diffusion face-swapping methods use strong image priors and high-fidelity generation, but iterative sampling and guidance can make inference expensive.DiffFace uses identity, semantic, and gaze guidance; DiffSwap uses 3D-aware masked diffusion and midpoint estimation.Source: §2 · p. 2
  • REFace [2] uses latent-diffusion inpainting and differentiable DDIM steps, while FaceAdapter separates identity, spatial, and attribute conditions.Their approaches also use perceptual and semantic guidance to support face or regional swapping.Source: §2 · p. 2
  • DreamID [52] is one-step but requires distilled initialization, whereas APPLE is a high-quality multi-step model with an order-of-magnitude larger training budget than ALADDIN.APPLE uses a diffusion teacher to produce pseudo-labels for student training; DreamID’s code and weights were unavailable for comparison.Source: §2 · pp. 2–3

3. Method

ALADDIN formulates swapping as one-step conditional generation, using the target as a noisy starting point and source identity as conditioning. Its objectives combine identity, realism, and target preservation, while teacher–student training and structured data groups provide reconstruction supervision and a tunable trade-off.

3. Method

  • The generator produces a face swap in one step from source and target images, treating the target as a sample at fixed noise level σgen = 0.2 before mapping it to a clean swap.It operates on frozen FLUX VAE latents; the generated image should carry source identity while retaining target attributes.Source: §3 · p. 3

3.1. Training Objectives

  • The identity objective minimizes cosine distance between generated and source identity embeddings, using a frozen face-recognition network.The loss reaches zero when embedding cosine similarity is 1.Source: §3.1 · p. 3
  • The realism objective combines adversarial training with reconstruction when data provides a ground-truth swap; identity loss alone can yield unrealistic images.The discriminator uses frozen diffusion-model activations rather than pixels or raw latents.Source: §3.1 · p. 3
  • Generated and real-image latents are corrupted at the same noise level, passed through the frozen diffusion model, and scored by discriminator heads on selected block activations.Trainable pooling logits aggregate the heads, while the diffusion model remains frozen and provides the generator’s prior without adding another pretrained model.Source: §3.1 · p. 3, Figure 2 · p. 4
  • Target-preservation losses constrain pose, expression, and background, using frozen pose, expression, and face-parsing estimators.The background mask complement retains hair, accessories, neck, clothing, and general background.Source: §3.1 · p. 4

3.2. Training Data Groups

  • For same-identity pairs, reconstruction provides direct supervision and improves image quality, but training on that group alone permits the generator to copy the target.The group needs no separate pose or expression loss because reconstruction already constrains those attributes.Source: §3.2 · p. 5
  • The triplet group uses two images of identity A and a frozen teacher’s pseudo-target carrying identity B with target attributes, enabling reconstruction supervision when source and target identities differ.The student is initialized from the teacher and trained on triplets; increasing its reconstruction weight trades identity transfer against realism and target preservation.Source: §3.2 · p. 5
  • Teacher training mixes cross-identity and same-identity groups, while the student trains on triplets with pseudo-targets generated on the fly by the frozen teacher.The student uses a freshly initialized discriminator.Source: §3.2 · p. 5

4. Experiments

ALADDIN is evaluated on FFHQ with metrics for realism, identity transfer, and target preservation, alongside an evaluation protocol designed to reduce identity-encoder bias. It compares favorably with one-step methods and matches the strongest multi-step baseline on several measures at one network evaluation, while ablations examine training data and discriminator design.

4.1. Experimental setup

  • Evaluation uses 1,000 FFHQ source–target pairs and measures realism, identity transfer, and target pose and expression preservation; WebFace–R50 scores identity independently of the encoders used to train ALADDIN.WebFace–R50 is used for evaluation only; the method uses Glint–R100 for identity conditioning and its identity loss.Source: §4.1 · pp. 5–6
  • Scoring with MS1M–IR-SE50 favors methods optimized against that encoder: REFace scores 0.64 there but 0.41 and 0.39 with two alternatives.Its own protocol ranks it first on identity transfer, while either alternative encoder allows several baselines to score higher.Source: §4.1 · p. 6, Table 1 · p. 6

4.2. Comparison with other methods

  • At 1 network evaluation, Studentλ=7 matches APPLE’s identity transfer at 0.40, FID (3.34 against 3.22), and expression (0.60 against 0.63), while improving pose (1.56 against 1.86); APPLE uses 28 steps.The comparison uses the same architecture.Source: Table 2 · p. 6, §4.2 · p. 7
  • Every ALADDIN model improves on MegaFS and E4S in all four metrics; Studentλ=3 also beats SimSwap, FaceDancer, and InfoSwap in all four.The Teacher and Studentλ=1 beat HiFiFace across all four metrics; only InSwapper and CSCS attain higher identity similarity, with costs elsewhere.Source: §4.2 · pp. 6–7
  • ALADDIN uses more than tenfold fewer training pairs than APPLE at matching quality, and its one-step teacher avoids offline pseudo-triplet generation.APPLE uses 1.04M teacher pairs and 0.24M student pairs; ALADDIN uses 0.10M and 0.015M, respectively.Source: §4.2 · p. 7
  • After triplet training, ALADDIN matches APPLE’s realism and target-attribute preservation at a fraction of its inference cost; increasing λrec favors realism and attribute retention over identity transfer.The comparison includes unusual hairstyles, large age gaps, glasses, tattoos, pose, and expression.Source: Figure 3 · p. 7, §4.2 · p. 8

4.3. Ablation study

  • Adding a quarter of same-identity pairs improves FID, pose, and expression over cross-identity-only training, while raising their share to one half degrades FID from 4.55 to 5.01.Identity similarity, pose, and expression remain essentially unchanged at the half-share setting; same-identity pairs do not reflect deployment.Source: §4.3 · p. 8
  • Replacing frozen teacher features with raw noised latents worsens FID from 5.50 to 20.46, pose from 2.20 to 2.82, and expression from 0.95 to 1.13.The raw-latent discriminator lacks a face prior and exerts almost no realism pressure, allowing heavily artefacted images.Source: §4.3 · p. 8

5. Conclusion and limitations

ALADDIN is a one-step face swapper initialized from a pretrained flow-matching model and trained adversarially; it matches the strongest multi-step baseline on identity transfer, FID, and expression while using fewer network evaluations and training pairs. The method retains field-wide failure cases, including degraded identity transfer for unusual source identities and large age gaps.

Conclusion

  • ALADDIN matches APPLE in identity transfer, FID, and expression, improves pose, and uses one network evaluation instead of 28 with an order of magnitude less training data.It is state-of-the-art among one-step methods.Source: §5 · p. 8
  • The discriminator must read intermediate activations of the frozen diffusion model rather than the latent itself to supply the realism prior adversarial training alone lacks.The pretrained flow model supplies that prior through both generator initialization and discriminator features, without a distillation objective.Source: §5 · p. 8

Limitations

  • Identity transfer degrades for unusual source identities and large age gaps; the weakest target-preservation model also loses tattoos.The authors describe resolving these cases as open work for face swapping as a whole.Source: §5 · p. 8

A. Implementation details

The experiments use a fixed FLUX prompt and guidance setting, with specified generator and discriminator noise levels and feature maps. Training and evaluation estimators differ for pose and expression, and triplet-group training trades identity transfer for realism and target-attribute preservation.

Implementation details

  • The generator uses noise level σgen = 0.2, while the discriminator reads four frozen-transformer feature maps at noise levels σ ∼U[0.01, 0.4].The model uses the fixed prompt “cinematic full head portrait” and guidance scale 4.5; noising latents before feature extraction made no clear difference to the identity–realism trade-off.Source: A. Implementation details · p. 12
  • Training pose and expression estimators, 6DRepNet and DECA, differ from the evaluation networks, so those metrics are not measured with the networks used to optimize the generator.Training uses one discriminator step per generator step and AdamW with learning rate 5 × 10−5.Source: A. Implementation details · p. 12

Training time

  • The teacher uses 100K image pairs and takes about 4.5 hours to train, while each student uses 15K pairs and takes about 50 minutes on 4xH100.Source: A. Implementation details · p. 12

Ablation runs

  • Quality saturates after roughly 70K pairs, so ablation checkpoints are selected after convergence to closely match identity similarity and compare FID, pose, and expression.The main teacher is reported at 100K pairs; cross-identity, equal-mixture, and raw-latent discriminator runs are reported at 70K, 80K, and 70K pairs, respectively. Triplet-group training improves realism and target attributes at the cost of identity transfer.Source: A. Implementation details · p. 12

B. Additional discussion of the identity-encoder bias in prior evaluation protocols

The previous evaluation protocol scored identity with MS1M–IR-SE50, producing conclusions that differ from the unbiased protocol. The paper notes that this encoder assigns higher identity similarity to almost every method than two other encoders, plausibly because its training corpus is smaller and less diverse.

Discussion of results under incorrect protocol, with MS1M–IR-SE50

  • Under the biased protocol, InfoSwap appears strong and REFace stronger still, but the unbiased protocol shows InfoSwap below Studentλ=3 and REFace below APPLE and several ALADDIN models.Under the biased protocol, no model is strictly better than REFace and InfoSwap.Source: B. Additional discussion of the… · p. 12

Comparison of the identity encoders

  • WebFace–R50 has more than 12 million training images and roughly 600,000 identities, compared with MS1M–IR-SE50’s 5,822,653 images and 85,742 identities; cited benchmark results report WebFace–R50 outperforming MS1M–IR-SE50.The paper regards WebFace–R50 as better suited for evaluation.Source: B. Additional discussion of the… · p. 12

Systematic over-estimation with MS1M–IR-SE50

  • MS1M–IR-SE50 assigns higher identity similarity than Glint–R100 and WebFace–R50 to almost every method; a plausible explanation is its smaller, less diverse training corpus and less discriminative embedding space.Source: B. Additional discussion of the… · p. 12

Discussion of results under incorrect protocol, with MS1M–IR-SE50

  • Table 4 reports results on 1,000 FFHQ pairs using MS1M–IR-SE50 for identity, while retaining the Table 2 protocol for FID, pose, and expression.The table orders methods by FID within each group.Source: Table 4 · p. 13

C. Extended discussion of qualitative results

The extended qualitative examples compare transfer of gender, skin colour, piercing, and source glasses. Results vary by attribute: ALADDIN and APPLE transfer piercing, while CSCS includes glasses from the source.

More examples

D. Ethical considerations

The authors acknowledge that one-step face swapping can lower the cost of producing non-consensual or deceptive imagery at scale, and describe identity-verification and film-production interests. They also specify the datasets and images used in their work.

D. Ethical considerations

  • One-step face swapping lowers the cost of producing non-consensual or deceptive imagery at scale.The authors state their interest is in identity verification, where realistic swaps can train and stress-test detection systems, and in film production.Source: D. Ethical considerations · p. 13
  • The authors train only on research datasets used in prior face-swapping work, and all qualitative examples use FFHQ images.Source: D. Ethical considerations · p. 13
  • Figures 4 and 5 use selected FFHQ pairs to probe gender, skin tone, piercing, source glasses, naturalness, age, and target-attribute preservation.Source: Figure 4 · p. 14, Figure 5 · p. 15

Related papers

Loading 2610.37503v1…