Source-linked AI summary
Towards the Automatic Anime Characters Creation with Generative Adversarial Networks
Yanghua Jin, Jiakai Zhang, Minjun Li, Yingtao Tian, Huachun Zhu, Zhihao Fang
TL;DR
Generating industry-standard anime facial images remains challenging because prior results are blurred or distorted. The paper combines a cleaner dataset with a DRAGAN-based training approach and tag estimation to build a stable, high-quality model. It also provides the pretrained model through an online website for anime character design.
Problem
Prior anime-face generation work had not produced promising results, leaving industry-standard facial-image generation challenging.
Method
The paper collects and cleans Getchu face images, estimates tags with Illustration2Vec, and trains a DRAGAN-based model using images without original tags.
Results
The combined efforts produce a stable, high-quality model for generating realistic anime facial images.
Takeaways & Limitations
A pretrained model is made accessible through an easy-to-use website for anime character design.
Takeaways & Limitations
Uneven label distributions limit some attribute combinations and bias FID when sampled-label priors differ from the empirical training distribution.
Abstract
from arXiv · showhide
Automatic generation of facial images has been well studied after the Generative Adversarial Network (GAN) came out. There exists some attempts applying the GAN model to the problem of generating facial images of anime characters, but none of the existing work gives a promising result. In this work, we explore the training of GAN models specialized on an anime facial image dataset. We address the issue from both the data and the model aspect, by collecting a more clean, well-suited dataset and leverage proper, empirical application of DRAGAN. With quantitative analysis and case studies we demonstrate that our efforts lead to a stable and high-quality model. Moreover, to assist people with anime character design, we build a website (http://make.girls.moe) with our pre-trained model available online, which makes the model easily accessible to general public.
1 Introduction
Prior GAN-based anime-face generators produced blurred or distorted results, leaving industry-standard facial-image generation challenging. This work addresses the problem through a cleaner dataset, a DRAGAN-based model, and tag-free training.
- Existing anime-face generation attempts, including DCGAN-based and conditional approaches, did not produce consistently promising results.
- The proposed model targets high-quality anime-face generation with a promising success rate.
- The contribution combines a clean Getchu dataset, a DRAGAN-based GAN model, and training from images without tags.
2 Related Works
GAN research established adversarial image generation and expanded it to convolutional, conditional, and text-guided settings. However, GAN optimization remains difficult because training can suffer from mode collapse and instability.
- GANs train a generator and discriminator simultaneously, with the discriminator distinguishing real samples from generated ones.
- Mode collapse can cause a generator to produce the same image repeatedly, motivating methods designed for more stable GAN training.
- DCGAN generates images from latent vectors, while conditional GANs use meaningful inputs such as class labels or encoded text.
3 Image Data Preparation
The paper constructs a cleaner anime-face dataset from Getchu, where images are relatively consistent in domain, quality, and alignment. It detects and manually filters faces, then estimates categorical tags because Getchu lacks metadata.
- Image-board datasets can contain substantial variation and noise, motivating a more consistent, clean, high-quality dataset for anime-face generation.
- Getchu character images are diverse in illustration style and theme while remaining within the character-image domain and being properly clipped and aligned.
- 3.1 Image Collection: The collection pipeline retrieves Getchu pages, detects faces, expands bounding boxes by 1.5x, and manually removes about 4% false positives and undesired images.
- 3.2 Tag Estimation: Because Getchu lacks image metadata, the paper uses Illustration2Vec to estimate 34 selected attributes from predicted anime-illustration tags.
- 3.3 Visualization: A t-SNE projection of 1500 sampled images places images with similar visual attributes close together.
4 Generative Adversarial Network
The paper reviews GAN objectives and training difficulties, then adopts DRAGAN for its stability, computational efficiency, and compatibility with simultaneous gradient descent. Labels are incorporated through conditional generation and discriminator-side multi-label classification.
- 4.1 Vanilla GAN: In the GAN objective, the discriminator distinguishes real data from generated samples while the generator seeks outputs that deceive the discriminator.
- 4.2 Improved training of GAN: GAN training can be unstable because generated and real distributions may have non-overlapping supports or lack a game equilibrium.
- 4.2 Improved training of GAN: DRAGAN is selected for presumable results, low computation cost, simultaneous gradient descent, and stability across several network architectures.
- 4.2 Improved training of GAN: DRAGAN adds a gradient penalty by sampling points near real images and forcing their discriminator-output gradients toward norm 1.
- 4.3 GAN with labels: For controllable generation, the generator receives noise z and a 34-dimension condition vector, while the discriminator predicts assigned tags with a multi-label classifier.
5 Experiments
The experiments use a DRAGAN-based architecture with distinct generator and discriminator designs, while addressing attribute imbalance through a specified conditioning strategy and dataset selection.
- Model architecture: The generator modifies SRResNet with 16 ResBlocks and three sub-pixel CNN layers for feature-map upscaling.
- Model architecture: The discriminator contains 10 ResBlocks, removes batch normalization to avoid mini-batch correlations in gradient-norm computation, and adds an attribute-classifier layer.
- Optimization: All experiments set λadv to 34 and λgp to 0.5, using Adam with β1=0.5, batch size 64, and an initially 0.0002 learning rate.
- Dataset and conditioning: Training uses images from games released after 2005, resized to 128*128 pixels, producing 31255 training images.
- Dataset and conditioning: Attribute labels are highly imbalanced, including 49 images with orange eyes versus 8861 with blue eyes.
- Dataset and conditioning: During training, hair and eye colors are sampled uniformly, while other attributes are independently assigned with probability 0.25.
5.2 Generated Results
The generated results show controllable visual consistency across conditions, while revealing that sample quality depends strongly on noise vectors and attribute coverage in the training data.
- Conditional generation: Fixing the noise while varying conditions produces images with similar major visual features, including face shapes and face directions.
- Conditional generation: The fixed-noise results provide evidence that visual concepts associated with labels generalize beyond brute-force memorization of training samples.
- Noise sensitivity: Noise vectors strongly affect final image quality: some produce good samples across conditions, whereas others more readily produce distorted images.
- Attribute coverage: Uneven label distributions lead some attribute combinations to produce poor images, especially combinations involving glasses or drill hair.
- Attribute coverage: Characters associated with well-learned attributes such as blonde hair and blue eyes appear attractive, while most glasses and drill-hair examples are distorted.
5.3 Quantitative Analysis
The evaluation examines attribute precision and FID, showing that color attributes are generally easier to learn than complex shape attributes, while rare colors can still achieve relatively high precision. FID is computed with Illustration2vec features because ImageNet features are unsuitable for anime illustrations.
- Attribute Precision: Color attributes are easier to learn than shape attributes such as “hat” and “glasses”.
- Attribute Precision: Similar color labels such as “white hair”, “silver hair”, and “gray hair” have unclear boundaries that reduce their precision scores.
- Attribute Precision: Rare colors including “orange eyes”, “aqua hair”, and “aqua eyes” achieve relatively high precision despite appearing in fewer than 1% of training samples.
- Attribute Precision: Complex attributes such as “hat”, “glasses”, and “drill hair” often produce distorted, difficult-to-identify images despite covering about 5% of training samples.
- FID: FID compares Gaussian approximations of real and generated feature distributions using Fréchet distance.
- FID: Illustration2vec replaces ImageNet’s Inception model for anime-image feature extraction, using 4096-dimensional vectors in the evaluation.
- FID: The reported FID comparison averages five calculation trials for each model, with the table comparing the proposed model against a baseline.
5.4 Website Interface
The website exposes the trained anime-face generator through a browser interface, allowing users to assign attributes while sampling unspecified ones from the training distribution. The implementation emphasizes smaller models and client-side generation.
- Implementation: The website converts the trained Chainer model into WebAssembly-based JavaScript using WebDNN and React.js.
- Model Size: A smaller generator benefits browser deployment because users must download the model before computation.
- Model Size: Replacing the DCGAN generator with SRResNet makes the model 4x smaller, reducing model-download time.
- User Interface: Users can assign attributes manually, while unassigned attributes are sampled from the training dataset’s empirical distribution.
- Performance: All generation computations run on the client side, taking about 6 ∼7s per image on average.
5.5 Super-Resolution
The super-resolution experiment addresses the difficulty of producing high-resolution anime faces from low-resolution training images by testing SRGAN. Higher adversarial loss increases sharpness but also introduces artifacts, and no satisfactory balance was found.
- Motivation: Low-resolution training images make direct generation of high-resolution facial images difficult.
- Method: SRGAN is selected because GAN-based super-resolution can add more high-frequency detail than MSE-only methods.
- Results: Increasing adversarial-loss weight makes upscaled images sharper but introduces more undesired artifacts.
- Limitation: Anime-style images are more sensitive to extra artifacts than real photos because their color and texture patterns are simpler and clearer.
- Limitation: The authors found no model balancing sharpness and artifact strength, so the super-resolution model was excluded from the website.
6 Conclusion
The work combines a clean dataset with GAN training strategies to generate realistic anime facial images and provides an online website service. The authors identify unresolved issues in evaluation with unbalanced labels and in increasing output resolution.
- Conclusion: The authors report successfully building a model that generates realistic anime facial images by combining a clean dataset with practicable GAN training strategies.
- Conclusion: An easy-to-use website service makes the trained model available online.
- Future Work: Improving the GAN model for unevenly distributed class labels remains an open direction.
- Future Work: FID can be biased when training labels are unbalanced because it measures only when sampled-label priors equal the empirical training distribution.
- Future Work: Improving the final resolution of generated images remains another direction, requiring more carefully designed and tested super-resolution models.
- Implications: The authors hope the work will stimulate further anime-style generative-modeling studies and help amateurs and professionals design characters.
8 Appendix
The appendix documents dataset image distributions, conditional-generation samples, database-query fragments, and browser inference-time comparisons. These materials cover dataset metadata, continuous attribute control, implementation details, and deployment performance.
- Implementation: The appendix includes SQL fragments selecting game identifiers, names, sale dates, and links for entries with non-null comike values, ordered by sale day.The query joins the displayed link prefix with each game's comike field.
- Dataset analysis: Figures 13 and 14 summarize available dataset images by release year and short-edge size.Figure 13 treats year=2030 as an undetermined release year.
- Conditional generation: Figure 15 shows randomly generated samples under different combinations of conditions.Discrete label-control variables assigned during training remain meaningful when used in a continuous setting.
- Deployment performance: Table 4 compares approximate inference time across environments, using WebGPU for Safari Technology Preview and WebAssembly for other browsers.The table notes that Firefox is better optimized with WebAssembly and faster than the other browsers.