Convolutional Network for Attribute-driven and Identity-preserving Human Face Generation

1. Introduction

• CNNs have been applied to high-level vision, image generation, and low-level vision tasks.The introduction positions face generation within broader CNN applications.Source: §1 · p. 1
• Earlier face-generation approaches include DG-IGN, which learns a latent graphic code from pictures and modifies it for pose and light; cGAN, which generates from noise and conditional attributes; and an attribute-conditioned VAE [8], which combines reference-image latents with attributes.DG-IGN requires many face images of one person across poses and lighting, generated by a 3D face model.Source: §1 · p. 1
• The paper frames identity-preserving attribute transfer as a needed capability because existing CNN face-generation models do not emphasize generated-face identity.The stated goal is to transfer attributes while keeping the reference face’s identity.Source: §1 · p. 1
• Face generation is formulated as optimization on VGG-Face, with attribute, identity, and regularization terms.VGG-Face was trained on a large-scale face-recognition dataset.Source: §1 · p. 1
• Gradient descent through VGG-Face generates a face from a blank image, with color transfer and an attribute mask added to improve output quality.Color transfer addresses mismatch with the reference face’s color space.Source: §1 · p. 2
• Experiments on LFW report realistic faces with given attributes while preserving the reference identity.The experiments also analyze which convolutional layers are used for the attribute and identity terms.Source: §1 · p. 2

2. Related works

• GANs, variational auto-encoders, and DRAW generate images using neural networks trained with loss functions, while this work formulates generation as optimization.The cited approaches include adversarial training, Gaussian latent variables, and attention-based patch trajectories.Source: §2 · p. 2
• Compared with 3D attribute-transfer systems, this model operates on 2D images without constructing a 3D model.The related systems include facial-expression transfer from 3D scans and face transfer from photo collections.Source: §2 · p. 2
• Unlike CNN visualization and reconstruction methods [25] [26], this work combines VGG-Face identity and attribute terms to preserve image content while modifying details.Spatial masks and color transforms are also used to improve generation quality.Source: §2 · p. 2

3.1. Perceptual loss

• VGG-Face perceptual loss compares convolutional features rather than requiring pixel-level equality, using its first five convolutional layers.The paper leaves deeper layers out because of their poor image-reconstruction performance.Source: §3.1 · pp. 2–3

3.2. Model and algorithm

• The guided set combines attribute-matched LFW faces selected for pose and content similarity to the reference, and their weighted perceptual losses form the attribute loss.Pose similarity uses 68 facial landmarks; content similarity uses semantic perceptual loss.Source: §3.2 · pp. 3–4
• The target face minimizes attribute and identity losses together, with total variation regularization for smoothness; gradient descent updates the image.The identity term compares the target with the reference, while the attribute term uses a weighted set of attribute-matched faces similar to the reference.Source: §3.2 · pp. 4–5
• Identity and attribute losses fall before plateauing, while pixel error stabilizes above zero because some attributes change from the reference.Figure 2 tracks squared pixel error, identity loss, and attribute loss over iterations.Source: §3.2 · p. 4, Figure 2 · p. 5

3.3. Improvements

• Spatial attribute masks restrict attribute loss to relevant image areas, reducing unintended changes to other attributes when similar training faces are scarce.Masks are built from associated facial landmarks and expanded convex hulls; masks for multiple attributes are combined by union.Source: §3.3 · p. 5
• A learned linear color transform maps generated-face pixels into the reference face’s YCbCr color space to improve color quality.The transform matrix is fit using sampled pixels from the central parts of the generated and reference faces.Source: §3.3 · pp. 5–6

4. Experiments

• Experiments use LFW faces aligned to 68 landmarks, resized to 224 × 224, and horizontally flipped for augmentation.Low-resolution faces can appear blurry after resizing; about 10% of attribute labels were incorrect, and faces with wrong labels or blurry attributes were excluded.Source: §4 · p. 6, §4.1 · p. 6

4.1. Attribute based image generation

• The model generates faces with glasses, mouth, nose, and smile attributes while preserving the reference identity.Figure 4 pairs reference faces on the left with generated faces on the right.Source: §4.1 · p. 6, Figure 4 · p. 7

4.2. Generate images with given guided images

• Given guided faces, the model transfers mouth attributes to target faces; Figure 5 shows the guide on the left and generated face on the right.Source: §4.2 · p. 6, Figure 5 · p. 7

4.3.1. Images generated from di

• conv2_1 gives the highest image quality but the least conspicuous attribute, whereas conv3_2 gives the poorest quality but a conspicuous attribute.The study chooses conv3_1 as a tradeoff between image quality and attribute salience.Source: §4.3.1 · p. 7, Figure 6 · p. 7, Table 1 · p. 8

4.3.2. Influence of the TV regulariser

• γ = 0 produces noisy, low-quality faces, while γ = 1 over-smooths faces and blurs the glasses outline.The TV regularizer addresses noise and mosaics in generated images; γ controls its contribution to the loss.Source: §4.3.2 · pp. 7–8, Figure 7 · p. 8

• The method formulates attribute-driven face generation as an optimization problem with semantic perceptual loss.Source: §5 · p. 8
• Spatial masks and color transforms are introduced to improve face-generation quality.Source: §5 · p. 8
• Gradient descent solves the optimization problem defined on the VGG-Face network.Source: §5 · p. 8
• Experiments show realistic faces with given attributes while preserving the reference identity.Source: §5 · p. 8
• Future work will incorporate CNN learning to improve the approach's efficiency and effectiveness.Source: §5 · p. 8