Source-linked AI summary
Generating Handwritten Chinese Characters using CycleGAN
Bo Chang, Qiong Zhang, Shenyi Pan, Lili Meng
TL;DR
Generating personalized Chinese handwriting is challenging because the character inventory is large and unpaired training is particularly relevant. The paper maps printed fonts to handwritten styles with DenseNet CycleGAN and reports efficacy on CASIA and Lanting calligraphy data.
Problem
Personalized Chinese handwriting generation must address a very large character inventory, and the task has received limited study with unpaired training data.
Source: §1 · pp. 1–2Method
Results
Takeaways & Limitations
Content accuracy and style discrepancy provide proposed metrics for assessing generated-character quality.
Source: §1 · p. 2DenseNet and ResNet generators fail to learn some features of Wang Xizhi’s semi-cursive style, including the simplification of two strokes into one.
Source: §4.5 · p. 7
Abstract
from arXiv · showhide
Handwriting of Chinese has long been an important skill in East Asia. However, automatic generation of handwritten Chinese characters poses a great challenge due to the large number of characters. Various machine learning techniques have been used to recognize Chinese characters, but few works have studied the handwritten Chinese character generation problem, especially with unpaired training data. In this work, we formulate the Chinese handwritten character generation as a problem that learns a mapping from an existing printed font to a personalized handwritten style. We further propose DenseNet CycleGAN to generate Chinese handwritten characters. Our method is applied not only to commonly used Chinese characters but also to calligraphy work with aesthetic values. Furthermore, we propose content accuracy and style discrepancy as the evaluation metrics to assess the quality of the handwritten characters generated. We then use our proposed metrics to evaluate the generated characters from CASIA dataset as well as our newly introduced Lanting calligraphy dataset.
1. Introduction
Personalized Chinese handwriting generation must address a very large character inventory with relatively few training examples. The paper frames the task as mapping printed fonts to a personalized handwritten style and proposes DenseNet CycleGAN, evaluation metrics, and tests on CASIA and Lanting.
1. Introduction
- Chinese font personalization must cover at least 3,000 commonly used characters, motivating automatic generation from a relatively small set of training characters.Chinese has more than 80,000 logographic characters.Source: §1 · p. 1
- Earlier stroke-based methods focus on local character parts, while paired-data methods such as zi2zi face the difficulty of collecting many matched handwritten examples.The paper argues that unpaired data better fit handwriting samples that may lack labels and vary across repeated writings.Source: §1 · p. 2
- The paper formulates generation as mapping an existing printed font to a personalized handwritten style and proposes DenseNet CycleGAN for that task.The proposed approach is evaluated on both the CASIA dataset and the newly introduced Lanting calligraphy dataset.Source: §1 · p. 2
- Content accuracy and style discrepancy are proposed as metrics for assessing the quality of generated characters.Source: §1 · p. 2
2. Related work
Prior Chinese character-generation work includes stroke-based construction and font transformation, while image-style-transfer methods face speed, quality, and cross-style content challenges. GAN approaches include paired-data translation and CycleGAN's unpaired translation.
2.1. Chinese character handwriting generation
- Stroke-based systems construct characters from hierarchical components, whereas zi2zi treats each character as a whole but requires paired source and target examples.Other work includes personalized TrueType font generation and stroke-correspondence methods for shape morphing.Source: §2.1 · p. 2
2.2. Image style transfer
- Descriptive style transfer iteratively updates each image, making it time-consuming at scale, while generative methods are faster but usually produce poorer transfer results.For Chinese characters, defining content loss across styles is difficult because stroke positions and angles can differ substantially.Source: §2.2 · pp. 2–3
2.3. Generative adversarial networks
- GANs model generation through competing generator and discriminator networks; conditional GANs and pix2pix use input conditions and paired related images for image translation.Zi2zi adapts pix2pix for Chinese fonts but uses paired character images, which are difficult to obtain for personalized handwriting.Source: §2.3 · p. 3
- CycleGAN learns image translation without paired examples by training two mappings with adversarial and cycle-consistency losses.Its default generator uses ResNet, while its discriminator uses a PatchGAN classifier.Source: §2.3 · p. 3
3. Our method
The method learns font-to-font mappings from unpaired source and target characters using CycleGAN, replacing its ResNet transfer blocks with DenseNet blocks. The generator encodes source images, transfers their features to the target style, and decodes them; adversarial and cycle-consistency losses form the training objective.
3. Our method
- Training uses unpaired source- and target-font samples to learn mappings between the two domains.The objective follows CycleGAN’s image/font transformations without paired examples.Source: §3 · p. 3
- The generator encodes a source-font image, transfers its features into a target-style representation, then decodes the result.The mapping takes an image from the source domain rather than random noise as input.Source: §3 · p. 3, Figure 3 · p. 4
- A 70×70 PatchGAN discriminator assesses generated target-domain images against true images.The reverse mapping and its discriminator are defined similarly.Source: §3 · p. 3
- Adversarial losses match generated-image and target-domain distributions, while cycle consistency requires cyclic transformations to return the original image.The cycle-consistency term acts as a regularizer controlled by λ; the full objective sums adversarial losses for both mappings and cycle-consistency losses.Source: §3 · pp. 3–4
- DenseNet CycleGAN replaces CycleGAN’s ResNet transfer blocks with DenseNet blocks to generate handwritten Chinese characters.DenseNet blocks receive feature maps from preceding blocks through concatenation.Source: §3 · p. 4
4. Experiments
The experiments evaluate handwritten-character generation on CASIA-HWDB and calligraphy generation on the Lanting dataset, using complementary measures of content and style. More training data improves both reported metrics, while the generated outputs capture recognizable handwriting and calligraphic style, with remaining limits on semi-cursive strokes.
4.1. Datasets
- Experiments use HWDB1.1 file HW252 (1252-c.gnt) as target style with SIMHEI as source, and Wang Xizhi’s 324-character semi-cursive Lantingji Xu with SIMKAI as source.Each HWDB1.1 file contains about 3,000 isolated grayscale Chinese-character images by one writer; images are resized to 128 × 128 pixels, with no other preprocessing.Source: §4.1 · p. 5
4.2. Performance metrics
- Content accuracy uses HCCR-GoogLeNet recognition to assess character content; style discrepancy compares target and generated style representations, with lower values indicating better style.The style discrepancy uses input to HCCR-GoogLeNet’s Inception module 3 as its feature layer.Source: §4.2 · pp. 5–6
4.3. Implementation details
- DenseNet-5 and ResNet-6 transfer modules have roughly the same number of parameters; training images are resized to 128 × 128, with λ = 10 and Adam at batch size 1.The learning rate is 0.0002 for the first 100 epochs, then linearly decays to zero over the next 100.Source: §4.3 · p. 6
4.4. Handwritten characters results
- Generated HW252 characters are recognizable, with clear backgrounds and stroke composition and blurry boundaries resembling the writer’s style.The examples compare SIMHEI source characters, ResNet-6 and DenseNet-5 outputs, and HW252 ground truth.Source: §4.4 · p. 6, Figure 5 · p. 7
4.5. Calligraphy results
- On Lanting, both generators capture Wang Xizhi’s overall style; DenseNet tends to make fewer missing- or extra-stroke errors, yet both miss the semi-cursive stroke merger.Generated examples outside Lanting are also reported as satisfactory.Source: §4.5 · pp. 6–7
4.4. Handwritten characters results
- For rA and rB greater than 10%, top-5 content accuracy is always greater than 80%; both metrics improve with more training data, and ResNet-6 and DenseNet-5 are comparable.Reported style discrepancy ranges from 503.77 to 3006.03, described as low on the spectrum.Source: §4.4 · p. 6, Table 3 · p. 7, Table 4 · p. 8
4.6. Comparison with neural style transfer
- Neural style transfer outputs retain a style almost identical to SIMHEI, have blurry backgrounds, and look worse overall than the paper’s method.The comparison uses VGG-19 with two SIMHEI content characters and 30% of HW252 as style images.Source: §4.6 · pp. 7–8, Figure 8 · p. 8
5. Conclusion
The paper frames Chinese handwritten character generation as mapping printed fonts to personalized handwriting, evaluates DenseNet CycleGAN on CASIA and Lanting, and reports superior or on-par performance.
5. Conclusion
- DenseNet CycleGAN uses DenseNet in the CycleGAN generator to improve Chinese handwritten character generation quality.The method is compared with original CycleGAN using ResNet blocks and Neural style transfer.Source: §5 · p. 8
- The method is evaluated on CASIA and the newly introduced Lanting calligraphy dataset using content accuracy and style discrepancy metrics.Source: §5 · p. 8
- Experiments demonstrate the method’s efficacy, with superior or on-par performance against the compared approaches.Source: §5 · p. 8