PaperlayerView exact revision on arXiv

Source-linked AI summary

MirrorPPR: Exemplar-Based Portrait Photo Retouching

Zhihong Liu, Zheng Li, Jiachun Jin, Siqi Kou, Yitao Jian, Fengpei Yu, Zhijie Deng

arXiv:2606.29308v1cs.CV

TL;DR

Structural portrait retouching requires transferring subtle, localized edits that text and existing exemplar-based methods struggle to express or detect. MirrorPPR introduces a dedicated extraction, transfer, and self-augmentation framework, and experiments show state-of-the-art retouching quality and identity preservation.

  • Problem

    Structural portrait retouching involves subtle facial and body adjustments that text-guided editing and existing exemplar-based models struggle to perceive and transfer.

  • Method

    MirrorPPR extracts subtle operations from exemplar pairs, transfers them through a pretrained diffusion transformer, and uses self-augmentation to address cross-identity misalignment.

  • Results

    MirrorPPR achieves state-of-the-art performance over existing baselines in both retouching quality and identity preservation.

  • Takeaways & Limitations

    MirrorPPR establishes exemplar-based structural portrait retouching as a task supported by dedicated operation extraction, aligned training, and a large-scale dataset.

  • Takeaways & Limitations

    Directly reusing exemplar pairs can cause spatial shortcut learning and impair generalization across identities, layouts, and scales.

Abstract

from arXiv · show

While text-guided image editing has made remarkable progress, it remains limited in structural portrait retouching. Textual descriptions struggle to convey fine-grained changes to facial features and body proportions. To address this gap, we introduce Exemplar-Based Portrait Photo Retouching, where the model is given an exemplar pair and tasked with inferring and applying the same retouching operations to a new query image. Existing exemplar-based editing methods primarily focus on tasks with pronounced visual transformations. In contrast, structural portrait retouching involves extremely delicate and localized modifications, making accurate extraction and transfer of these edits challenging. To tackle this, we propose MirrorPPR, a novel framework designed to capture and transfer subtle structural retouching operations. Our method uses a Retouching Operation Extractor to capture the subtle differences from the exemplar pair. The extracted representations are then injected into a pre-trained Diffusion Transformer (DiT) through a connector and Low-Rank Adaptation (LoRA) modules. Furthermore, constructing perfectly aligned cross-identity training pairs is severely hindered by operation misalignment. To overcome this, we propose an advanced data self-augmentation paradigm that ensures strictly aligned retouching operations. To alleviate data scarcity and support this novel task, we introduce MirrorPPR47M, a large-scale dataset with over 47 million retouched pairs. By structuring the dataset into simulated and professional subsets, we enable progressive curriculum learning to smoothly optimize the network. Extensive experiments demonstrate that MirrorPPR significantly outperforms existing baselines in both retouching quality and identity preservation. The project page is available at https://sjtu-deng-lab.github.io/MirrorPPR.

1 Introduction

Structural portrait retouching is difficult for text-guided methods because natural language cannot precisely specify subtle, localized geometric edits, motivating exemplar-based transfer. MirrorPPR addresses this challenge with operation extraction, DiT conditioning, self-augmented training, and the 47M-pair MirrorPPR47M dataset.

  • Motivation: Text-guided methods struggle with structural portrait retouching because language cannot quantitatively specify fine-grained edits’ spatial scale, direction, and magnitude.Structural adjustments include refining jawlines, enlarging eyes, and modifying body proportions.
  • Motivation: Exemplar-based editing conveys retouching intent through before-and-after pairs, but subtle structural operations remain difficult to extract and transfer.Unlike general editing tasks, structural retouching involves remarkably subtle transformations.
  • MirrorPPR Framework: MirrorPPR extracts retouching differences from exemplar pairs and injects the resulting representation into a pre-trained DiT to guide transfer to query images.The framework uses a Retouching Operation Extractor, a connector, and joint LoRA fine-tuning after extractor pre-training.
  • Data Self-Augmentation: Synchronous spatial augmentations construct self-augmented training quadruplets with strictly aligned operations, removing the need for cross-identity paired examples.The method applies randomized spatial augmentations to source and target exemplars synchronously.
  • Dataset and Results: Over 47 million retouched pairs form MirrorPPR47M, whose simulated and professional subsets support curriculum learning from pronounced deformations to subtle authentic edits.The dataset addresses scarcity and enables progressive optimization for structural retouching.
  • Dataset and Results: MirrorPPR significantly outperforms existing baselines, achieving state-of-the-art retouching quality and identity preservation.The reported evaluation covers both retouching quality and identity preservation.

2 Related Work

Related work progresses from text-guided image editing toward exemplar-based editing to reduce semantic ambiguity, while portrait retouching remains limited by fine-grained structural-editing challenges and insufficient training data.

  • Text-guided Image Editing: Diffusion-based text-guided editing supports general edits, non-rigid editing, image composition, style transfer, and other complex tasks.P2P and InstructPix2Pix address general edits, MasaCtrl enables tuning-free non-rigid editing, and multi-reference methods support composition and style transfer.
  • Exemplar-Based Image Editing: Exemplar-based editing has emerged to address text’s semantic ambiguity, progressing from inversion-based methods to in-context learning for non-rigid and heterogeneous tasks.Early approaches map visual examples to textual embeddings or discrete instructions, while EditTransfer and VIRAL advance in-context learning.
  • Portrait Photo Retouching: Traditional portrait retouching emphasizes global appearance enhancements, whereas generative structural editing still struggles with fine-grained localized adjustments and suitable training data is lacking.Examples of appearance-level processing include color grading and skin smoothing; the passage identifies insufficient suitable training data as a major bottleneck.

3 Method

MirrorPPR extracts subtle retouching operations from exemplar pairs and transfers them to query portraits through a connector and LoRA-enhanced DiT. Self-Augmentation preserves operation consistency while preventing pixel-level shortcuts and cross-identity misalignment.

  • Framework: MirrorPPR combines a Retouching Operation Extractor with a pre-trained DiT, transferring exemplar retouching operations through a dedicated connector and LoRA modules.The extractor captures subtle differences, while the DiT applies them to the query image.
  • Retouching Operation Extractor: The extractor uses a frozen MAE to preserve local spatial structures and a trainable R-Former to extract subtle retouching operations.R-Former query tokens interact with MAE patch features through self-attention to represent the edit.
  • Extractor Pre-training: An auxiliary reconstruction task pre-trains the extractor by injecting a compact edit embedding into query patch tokens and reconstructing the retouched result.After pre-training, the temporary MLP and ViT decoder are discarded.
  • Data Self-Augmentation: Cross-Identity training suffers operation misalignment from differences in shot scale, pose, occlusion, and composition, producing ambiguous supervision.Self w/o Aug avoids this misalignment but enables coordinate-wise copying because exemplar and query pairs are pixel-aligned.
  • Data Self-Augmentation: Self-Augmentation applies the same random spatial transformation A to source and target images, preserving exact operation consistency while breaking spatial coupling.The augmentation includes scaling, cropping, rotation, and horizontal flipping, with Xq = A(Xs) and Yq = A(Xt).

4 MirrorPPR47M Dataset

MirrorPPR47M is a large-scale portrait dataset for structural reshaping across facial features, face contours, and body proportions, designed to support easy-to-hard curriculum learning. It combines simulated and professional retouching subsets generated through controlled geometric operations and commercial retouching.

  • Dataset Overview: MirrorPPR47M covers structural reshaping of facial features, face contours, and body proportions across diverse portrait shot scales.Its easy-to-hard curriculum addresses the challenge of learning subtle real-world retouching operations.
  • Dataset Composition: 30,171 FFHQ images form the Simulated Retouching Subset, while 3,789 4K-8K PPR10K portraits form the Professional Retouching Subset.Both subsets are selected from high-resolution images using strict filtering criteria based on head pose, facial occlusion, and portrait area ratio.
  • Simulated Retouching Generation: 808,439 simulated retouched pairs are synthesized with Landmark-Guided Local Warping using eight facial operation types, opposite directions, and one-to-eight random operations per identity.The LLW algorithm is designed to produce pronounced geometric deformations.
  • Professional Retouching Generation: 46,642,845 professional retouched pairs are generated by applying random combinations of one-to-seven operations through a commercial retouching API.The 27 professional operations include 18 facial-feature, 4 face-shape, and 5 body-proportion operations.

Appendix A.4 provides more detailed operation taxonomy and statistics.

The appendix details a self-augmentation pipeline that generates operation-aligned yet spatially decoupled training quadruplets, expanding each original pair into approximately 13.3 variations. It also describes progressive training from simulated to professional retouching data to capture complex structural operations.

  • Data self-augmentation: The pipeline generates spatially decoupled yet operation-aligned training quadruplets (Xs, Xt, Xq, Yq).It follows the Self-Augmentation paradigm described in Section 3.4.
  • Data self-augmentation: Portraits are cropped to approximately 4 MP using facial bounding boxes and human body masks to preserve subject completeness.The cropping procedures are provided in Appendix A.3.
  • Data self-augmentation: Mirroring, rotation, and dynamic cropping with varied offsets place query-pair operations at different coordinates and scales from the exemplar pair.These synchronized spatial augmentations help decouple spatial appearance while preserving the retouching operation.
  • Data self-augmentation: 13.3 spatially decoupled variations are generated from each original pair, preventing shortcut learning and expanding the training data.The expansion factor is approximately 13.3 per original pair.
  • Progressive training: The extractor is pre-trained on simulated data, adapted on professional data, and finally fine-tuned exclusively on professional data.This progressive recipe targets fundamental structural variations before intricate real-world retouching subtleties and complex operations.

5 Experiments … A.2 Landmark-Guided Local Warping (LLW) Algorithm

The paper evaluates MirrorPPR on cross-identity simulated and professional benchmarks, showing strong retouching quality, identity preservation, and user preference, while ablations validate self-augmentation. It also details the MirrorPPR47M data construction, filtering criteria, and landmark-guided warping pipeline used to synthesize aligned retouching pairs.

  • 5.2.1 Quantitative Analysis: MirrorPPR-Pro achieves PSNR 32.65, SSIM 0.927, and Face Similarity 0.960 on the challenging ProPortrait-500 benchmark.MirrorPPR-Face and MirrorPPR-Pro consistently perform strongly across both benchmarks and evaluation metrics.
  • 5.2.3 User Study: 79.0% of users prefer MirrorPPR-Pro for transferring retouching while preserving identity and unrelated regions.Participants selected the anonymized candidate that best balanced operation transfer, identity preservation, and preservation of unrelated areas.
  • 5.4 Ablation Study: Self-Augmentation matches Cross-Identity on SimFace-100, outperforms it across all ProPortrait-500 metrics, and converges faster.The advantage is strongest when professional cross-identity data contains diverse identities, poses, compositions, and locally applicable operations that are difficult to align.
  • 6 Conclusion: MirrorPPR integrates a Retouching Operation Extractor with a Diffusion Transformer and uses Self-Augmentation plus MirrorPPR47M’s over 47 million pairs for progressive curriculum learning.The conclusion identifies these components as responses to operation misalignment and data scarcity.
  • A Dataset Details: MirrorPPR47M construction covers filtering, Landmark-Guided Local Warping, retouching-operation statistics, and data self-augmentation.The dataset-details section describes the overall scope of the construction pipeline.
  • A.1 Data Filtering Criteria: Simulated images retain FFHQ samples with all head-pose angles within ±15°, while discarding severe blur, poor exposure, high noise, or critical-region occlusions.Professional images from PPR10K are restricted to single-person portraits with detected portrait areas of at least 240,000 pixels.
  • A.2 Landmark-Guided Local Warping (LLW) Algorithm: LLW detects 468 MediaPipe facial landmarks, displaces operation-specific moving points while fixing anchors, and computes target coordinates from manually defined deformation rules.Similarity MLS maps target-grid pixels to source coordinates, followed by bilinear interpolation and mask-based feathered alpha blending.

A.3 Data Processing and Self-Augmentation · A.4 Taxonomy and Statistics of Retouching Operations

MirrorPPR standardizes portrait pairs and applies self-augmentation that preserves the main subject while generating about 13 augmented views per pair. Its taxonomy spans simulated directed operations and fine-grained professional retouching across facial features, face shapes, and body proportions.

  • A.3 Data Processing and Self-Augmentation: All images are standardized to approximately 4 MP, with height and width constrained to multiples of 16.This processing adapts images to the model architecture and improves training efficiency.
  • A.3 Data Processing and Self-Augmentation: Simulated images are directly resized, while professional images are YOLO-cropped using portrait bounding boxes expanded 10% on every side before resizing.The resulting processed images serve as base exemplar pairs.
  • A.3 Data Processing and Self-Augmentation: Self-augmentation applies randomized horizontal flipping to both images in each exemplar pair.Horizontal flipping mirrors images left-to-right.
  • A.3 Data Processing and Self-Augmentation: Dynamic cropping generates multiple pair versions with extreme aspect ratios ranging from 1:3 to 3:1.Facial bounding boxes from FFHQ and YOLO body masks ensure augmented views retain the main portrait subject.
  • A.3 Data Processing and Self-Augmentation: Augmentation produces an average of 13.3 variations per simulated pair and 13.4 variations per professional pair.The cropping constraints preserve the main portrait subject during augmentation.
  • A.4 Taxonomy and Statistics of Retouching Operations: MirrorPPR47M covers structural retouching operations spanning facial features, face shapes, and body proportions.This taxonomy is designed to provide comprehensive structural portrait-retouching coverage.
  • A.4 Taxonomy and Statistics of Retouching Operations: The simulated subset uses 8 LLW-based operation types, each with two directions, yielding 16 directed operations; each base type represents 12.5%.The number of operations per exemplar pair is uniformly distributed from 1 to 8, with each count also representing 12.5%.
  • A.4 Taxonomy and Statistics of Retouching Operations: The professional subset contains 27 fine-grained operations: 18 facial-feature, 4 face-shape, and 5 body-proportion operations.Each of the 22 facial and face-shape operations accounts for approximately 3.4% of sampled instances, while each body-proportion operation accounts for approximately 5.0%.

B Model Architecture Details

The framework’s trainable architecture comprises the R-Former, auxiliary ViT decoder, connector, and LoRA modules, with module-specific image preprocessing and a connector linking the R-Former to the frozen DiT.

  • Architecture: Trainable components mainly comprise the R-Former, auxiliary ViT decoder, connector, and LoRA modules.These are the framework’s principal trainable components.
  • Input Processing: Inputs are resized to approximately 2048×2048 pixels for the Retouching Operation Extractor, 512×512 for Qwen2.5-VL, and 1536 × 1536 for the VAE encoder.The VAE setting follows the recommended configuration of the Qwen-Image-Edit-2511 backbone.
  • Module Designs: The R-Former and ViT decoder use standard Vision Transformer architectures, while the R-Former includes internal learnable query tokens.The connector bridges the R-Former and frozen DiT, follows the Qwen2.5 LLM architecture, enables bi-directional attention, and uses an Enc-Proj design.

C Training Details

MirrorPPR trains Face and Pro variants with distinct staged optimization schedules. Face uses auxiliary-task pretraining before joint fine-tuning, while Pro applies progressive pretraining across simulated and professional data before professional-only fine-tuning.

  • MirrorPPR-Face: MirrorPPR-Face pre-trains the Retouching Operation Extractor for approximately 40,000 steps using auxiliary reconstruction, then jointly fine-tunes the entire framework for 150,000 steps.The ViT decoder is discarded before joint fine-tuning.
  • MirrorPPR-Pro: MirrorPPR-Pro pre-trains the extractor for approximately 60,000 steps on simulated data, then 40,000 steps on professional data.This staged schedule implements progressive curriculum learning.
  • MirrorPPR-Pro: MirrorPPR-Pro then jointly fine-tunes the entire framework exclusively on the professional subset for 150,000 steps.The final fine-tuning stage follows both extractor pretraining stages.

D Evaluation Details · E User Study · F More Ablation Studies

The evaluation details specify officially recommended inference configurations with query-matched output areas, while the user study compares MirrorPPR-Pro against representative baselines using anonymized candidate outputs and preference voting. The supplied passages do not describe the additional ablation studies.

  • D Evaluation Details: All evaluated models use their officially recommended inference configurations.These configurations are summarized in Table 10.
  • D Evaluation Details: Output resolution generally matches the query image, using areas equivalent to 1024×1024 pixels for SimFace-100 and 1.5K × 1.5K pixels for ProPortrait-500.The stated areas denote approximate pixel areas rather than fixed square dimensions.
  • E User Study: The user study evaluates 100 ProPortrait-500 samples and compares MirrorPPR-Pro with Nano Banana 2 and EditTransfer as category-specific representative baselines.Nano Banana 2 represents both multi-reference image editing and text-guided editing, while EditTransfer represents exemplar-based editing.
  • F More Ablation Studies: The supplied passages provide no substantive description of the more ablation studies.No ablation setup, result, or conclusion appears in the provided evidence.
  • D Evaluation Details: Table 10 reports inference configurations for all evaluated models, with “default” indicating that output size follows the input query image.The query-image areas are equivalent to 1024×1024 pixels on SimFace-100 and 1.5K × 1.5K pixels on ProPortrait-500.
  • E User Study: Each question presents an exemplar pair, a query image, and four anonymized outputs for candidate selection.Participants select the candidate that best matches the demonstrated retouching operation and edit strength while preserving identity and leaving unrelated backgrounds unaffected.
  • E User Study: Responses from 30 users are converted into each method’s preference as the percentage of total selections.The full results are reported in Table 11, and the web interface is shown in Figure 8.

F.1 Failure Analysis of the “Self w/o Aug” Strategy

Without spatial augmentation, the “Self w/o Aug” strategy learns edit locations as absolute coordinates rather than transferring retouching operations. It consequently alters irrelevant background regions and misses the query face, demonstrating severe spatial shortcut learning.

  • Failure mechanism: The “Self w/o Aug” strategy fails when spatial augmentation is removed.Figure 9 provides a qualitative failure case for this setting.
  • Failure mechanism: The model memorizes the exemplar edits’ absolute spatial coordinates, particularly the bottom-right position.The difference map reveals this spatial shortcut directly.
  • Failure consequence: It modifies irrelevant query background in the bottom-right and fails to edit the face located in the top-left.This mismatch shows that the operation is not transferred according to facial content.
  • Implication: The failure demonstrates severe spatial shortcut learning and validates the necessity of spatial augmentation.Spatial augmentation is therefore needed to prevent reliance on absolute edit locations.

F.2 Model Design Ablations · G More Qualitative Examples

Model-design ablations favor moderate settings—8 query tokens, a 4-layer R-Former, and LoRA rank 32—for balancing performance and efficiency. Additional qualitative comparisons show that MirrorPPR transfers delicate retouching operations more precisely than existing baselines, which exhibit various issues.

  • F.2 Model Design Ablations: F.2 Model Design Ablations: Ablations vary query-token count, R-Former depth, and LoRA rank while keeping other settings fixed on MirrorPPR-Face and SimFace-100.The default configuration uses Nquery = 8, a 4-layer R-Former, and LoRA rank r = 32.
  • F.2 Model Design Ablations: F.2 Model Design Ablations: Increasing learnable query tokens yields marginal metric gains but increases operation-condition length and training and inference overhead.The method therefore uses Nquery = 8 by default.
  • F.2 Model Design Ablations: F.2 Model Design Ablations: The 4-layer R-Former performs best, while 2 layers slightly reduce reconstruction quality and 8 layers decrease Face Similarity without further gains.The results suggest moderate depth is sufficient for extracting subtle retouching operations, whereas deeper extractors may introduce redundancy.
  • F.2 Model Design Ablations: F.2 Model Design Ablations: LoRA rank r = 8 underperforms, whereas r = 64 slightly improves PSNR and LPIPS but reduces Face Similarity and adds computational overhead.The default r = 32 provides the best trade-off.
  • G More Qualitative Examples: G More Qualitative Examples: Figures 10–13 compare MirrorPPR with baseline methods on SimFace-100 and ProPortrait-500.The comparisons further demonstrate the method’s ability to capture and transfer delicate retouching operations to query images.
  • G More Qualitative Examples: G More Qualitative Examples: Existing baselines suffer from various issues in qualitative comparisons, while MirrorPPR precisely transfers delicate retouching operations.Figure 9 separately analyzes “Self w/o Aug,” which overfits to the exemplar’s absolute edit position and fails on the query image.
Loading 2606.29308v1…