MaskGAN: Towards Diverse and Interactive Facial Image Manipulation

• Existing facial manipulation methods either use predefined attributes or provide limited freedom for interactive image editing.Source: §1 · p. 1
• MaskGAN uses semantic masks as an intermediate representation for flexible manipulation while preserving image fidelity.Users can specify facial component categories, shapes, and locations through modified masks.Source: §1 · p. 1
• Dense Mapping Network learns style mapping between target images and user-modified semantic masks, while EBST models user editing behavior.DMN includes an image generation backbone and spatial-aware style encoder.Source: §1 · pp. 1–2
• EBST introduces dual-editing consistency and improves robustness to varied user-input mask changes during inference.Its training uses a pretrained DMN, MaskVAE, and alpha blender sub-network.Source: §1 · p. 2
• MaskGAN outperforms other state-of-the-art methods on attribute transfer and style copy tasks.The evaluation covers two challenging facial manipulation tasks.Source: §1 · p. 2
• CelebAMask-HQ provides over 30,000 512×512 face images with fine-grained masks covering 19 facial component categories.The dataset was constructed to facilitate large-scale studies.Source: §1 · p. 2
• The paper presents MaskGAN, EBST, and CelebAMask-HQ as contributions intended to support interactive editing, robust mask manipulation, and geometry-oriented research.Source: §1 · p. 2

• Semantic-level face manipulation methods have produced impressive results using approaches such as inverse mappings, attribute disentanglement, and multi-domain translation.Source: §2 · p. 2
• Some geometry-level methods transfer facial attributes between exemplars, including approaches that exchange latent codes between two faces.The cited geometry-level methods address instance-level attribute transfer.Source: §2 · p. 3

Overall Framework

• MaskGAN uses a target image, target semantic mask, and user-modified source mask to synthesize a manipulated output face.The framework includes DMN, MaskVAE, and Alpha Blender, trained with EBST.Source: §3 · p. 3

Dense Mapping Network

• Dense Mapping Network combines a Spatial-Aware Style Encoder with an Image Generation Backbone to map target-image style onto corresponding regions of the source mask.The encoder uses both target-image appearance and target-mask spatial information before synthesis.Source: §3.1 · pp. 3–4, Figure 3 · p. 3

Editing Behavior Simulated Training

• MaskVAE models structural-prior manifolds, enabling smooth mask interpolation that supports EBST's locally perturbed mask inputs.EBST uses interpolated masks and an Alpha Blender to retain subject appearance and identity information.Source: §3.2 · p. 4, Figure 4 · p. 4
• EBST trains DMN with dual edited masks and blending, making the generator more robust to changes in user-modified masks during inference.The procedure alternates updates to the DMN and the combined model until convergence.Source: §3.2 · p. 4

Evaluation

• On facial attribute transfer and style copy, MaskGAN is reported to outperform other state-of-the-art methods.The supplied figure captions specifically discuss attribute preservation differences in smiling transfer and style copy.Source: Figure 5 · p. 5

Multi-Objective Learning

• The training objective combines conditional adversarial, feature-matching, and perceptual losses, while multi-scale discriminators support high-resolution image synthesis.The feature and perceptual weights are both set to 10 through cross validation.Source: §3.3 · p. 5

Dataset Construction

• CelebAMask-HQ contains 30,000 high-resolution face images labeled with semantic masks.The dataset is based on CelebA-HQ images from CelebA.Source: §4 · p. 5

Annotations

• Each 512 × 512 image has hand-annotated labels covering 19 facial components and accessories.Classes include skin, nose, eyes, eyebrows, ears, mouth, hair, hat, eyeglasses, and clothing.Source: §4 · p. 5

Label Size Selection

• The dataset uses 512 × 512 labels because annotating at 1024 × 1024 would be costly, while nearest-neighbor interpolation can extend labels without noticeable artifacts.The original CelebA-HQ images were 1024 × 1024.Source: §4 · p. 5

Quality Control

• Every segmentation mask undergoes quality-control checks, followed by multiple refinement rounds by annotators.The process is applied after manual labeling.Source: §4 · p. 5

Amodal Handling

• Partly occluded components are labeled by inferring hidden regions, while totally occluded components are left unannotated.This is the dataset's stated occlusion-handling policy.Source: §4 · p. 5

Dataset Comparison

• CelebAMask-HQ is compared with the Helen dataset through dataset statistics.The comparison is reported in Table 5.Source: §4 · p. 5

Attribute Transfer

• MaskGAN achieves relatively high classification and segmentation accuracy for smiling transfer, although it does not obtain the best FID score.StarGAN and ELEGANT have better FID, while Pix2PixHD-m has the best classification accuracy but inferior FID.Source: Table 2 · p. 6, §5.4 · p. 8
• MaskGAN provides plausible smiling-transfer results, whereas StarGAN fails in the smiling region, ELEGANT transfers smiling inconsistently, and Pix2PixHD-m has lower perceptual quality.SPADE has poor attribute-keeping ability in the visual comparison.Source: Figure 7 · p. 6, §5.4 · p. 8

Style Copy

• For style copy, MaskGAN transfers makeup and gender styles better visually than SPADE and Pix2PixHD-m, while SPADE obtains better segmentation accuracy.MaskGAN also has relatively high attribute classification accuracy compared with Pix2PixHD-m.Source: Figure 8 · p. 7, Table 3 · p. 7, §5.4 · p. 8

Ablation Study

• Editing Behavior Simulated Training improves robustness to structure changes in user-modified masks and strengthens attribute-keeping evaluation scores.The ablation attributes this improvement to reducing content-generation influence from structural changes during inference.Source: §5.5 · p. 8

Interactive Face Editing

• MaskGAN supports interactive geometry-level editing of facial-component shape, location, and category, including accessories, face and nose shape, and hair.The interactive results include a hair-color failure case when adding hair to a short-hair woman.Source: §5.6 · p. 8

Conclusions

• MaskGAN uses Dense Mapping Network and Editing Behavior Simulated Training to support geometry-oriented face manipulation through semantic masks.The framework targets flexible manipulation with fidelity preservation.Source: §6 · p. 8
• The framework is evaluated on attribute transfer and style copy, where it shows superior performance over other state-of-the-art methods.The conclusion summarizes both challenging evaluation tasks at the paper level.Source: §6 · p. 8
• CelebAMask-HQ contributes a large-scale, high-resolution face dataset with fine-grained mask annotations.The dataset is presented as an additional contribution alongside the manipulation framework.Source: §6 · p. 8
• Future work includes combining MaskGAN with image completion techniques to preserve details in regions without editing.This is the paper’s stated future direction.Source: §6 · p. 8

Network Architecture

• MaskGAN comprises MaskVAE, Dense Mapping Network, Alpha Blender, and Discriminator.Dense Mapping Network contains the Image Generation Backbone and Spatial-Aware Style Encoder.Source: A. Additional Implementation Details · p. 11

Spatial Feature Transform

• Spatial Feature Transform layers use more parameters to fuse different domains and are reported to work better than concatenation.The comparison is summarized quantitatively in Table 5.Source: B. Additional Ablation Study · p. 11

Editing Behavior Simulated Training

• Editing Behavior Simulated Training produces better color saturation and attribute-keeping quality for style copy.The visual comparison specifically mentions heavy makeup as an attribute example.Source: B. Additional Ablation Study · p. 11

Attribute Transfer

• Figures 14–17 show additional visual results for smiling attribute transfer against Pix2PixHD, ELEGANT, and StarGAN.The figures include models trained at 256 × 256 resolution, marked with an asterisk.Source: C. Additional Visual Results · pp. 11–12, Figure 14 · p. 13, Figure 15 · p. 14

EBST

• Figure 13 provides visual comparisons between training with and without Editing Behavior Simulated Training.Source: Figure 13 · p. 12

Ablation Study

• Table 5 reports a style-copy ablation using attribute-classification accuracy for Male, Heavy Makeup, and No Beard.Its train/test split differs from the main paper.Source: Table 5 · p. 12

Style Copy

• Figures 18–21 show additional visual results for style copy compared with modified Pix2PixHD.Source: C. Additional Visual Results · p. 12, Figure 18 · p. 17, Figure 19 · p. 18

Interactive Interface

• The accompanying interface lets users edit facial-component shapes or add accessories by manipulating the semantic segmentation mask.Source: C. Additional Visual Results · p. 12