Artistic Glyph Image Synthesis via One-Stage Few-Shot Learning

1 INTRODUCTION

• Earlier methods focused on shape or texture, while MC-GAN stacked separate shape-generation and texture-decoration models and was limited to 26 Latin capital letters.Optimization-based texture transfer also required reference outlines and time-consuming online optimization.Source: §1 · p. 2
• AGIS-Net uses two encoders and parallel decoders to transfer shape and texture styles together in one stage from a small number of reference samples.The decoders share content and style features, reducing the number of model parameters.Source: §1 · p. 2
• Experiments on English and Chinese glyph-image datasets show that AGIS-Net markedly outperforms existing approaches in synthesizing realistic, high-quality stylized glyph images.The introduction also reports superiority over other state-of-the-art methods.Source: §1 · p. 2
• A local texture refinement loss targets high-quality local patches, while contextual loss improves realism at the feature level.The local loss refines image details through adversarial training.Source: §1 · p. 2
• The Chinese dataset includes 1,571,940 glyph images from 2,460 synthetic artistic font styles and 256,410 images rendered from 35 professional-designed fonts.Each professional-designed font has 7,326 characters.Source: §1 · p. 2

2.1 Font Generation

• Font-generation work used font manifolds for interpolation [Campbell and Kautz 2014], stroke learning for Chinese handwriting [Lian et al. 2018], and geometric features [Balashova et al. 2019].Other approaches used deep networks for English glyphs [Baluja 2016], image analogy [Upchurch et al. 2016], image translation [Lyu et al. 2017], and structure-guided Chinese font synthesis [Jiang et al. 2019].Source: §2.1 · p. 2

2.2 Style Transfer

• Style-transfer methods include CNN-based stylization [Gatys et al. 2015; Johnson et al. 2016], arbitrary transfer [Huang and Belongie 2017], and patch-based transforms [Li et al. 2017].Glyph texture work used optimization [Yang et al. 2017; Men et al. 2018] or deep learning [Yang et al. 2019], while other work explicitly disentangled content and style [Tenenbaum et al. 2000; Wang et al. 2016; Gonzalez-Garcia et al. 2018; Kazemi et al. 2019].Source: §2.2 · p. 3

2.3 Generative Adversarial Networks

• GAN-based image translation included Pix2Pix [Isola et al. 2017], multimodal BicycleGAN [Zhu et al. 2017b], mode-seeking regularization [Mao et al. 2019], and unpaired cycle-consistency methods [Lee et al. 2018; Zhu et al. 2017a].MC-GAN [Azadi et al. 2018] connected two networks in series for shape-style and texture-style transfer.Source: §2.3 · p. 3

3 METHOD DESCRIPTION

• A content reference and a randomly sampled subset of same-style, different-content references condition generation; sampling subsets offers varied training combinations and avoids feeding all references at once.The authors also cite smaller model size as a reason for subset sampling in writing systems such as Chinese.Source: §3 · p. 3
• Two encoders extract content and style features, while parallel decoders generate shape and texture, sharing encoded features to reduce model parameters.Pre-training precedes few-shot fine-tuning to a specific artistic style.Source: §3 · p. 3

3.1 Network Architecture

• Skip connections combine features from both encoders at each shape-decoder layer, then add shape-decoder features to the texture decoder and its final output.This architecture exploits features at different scales and is intended to give shape style greater attention because glyph shape varies more than color and texture.Source: §3.1 · p. 4, Figure 3 · p. 4
• Shape and texture outputs go to separate PatchGAN-like discriminators, while a third local discriminator processes style inputs and generated texture patches.The shape discriminator's real samples are grayscale versions of the texture discriminator's real samples.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 Objective Function

• The objective combines adversarial, L1, contextual, and local texture refinement losses; L1 and contextual terms apply to grayscale and texture outputs.During fine-tuning, L1 and contextual weights are zero for characters without ground-truth images, though those characters remain useful for adversarial training.Source: §3.2 · pp. 4–5
• The local discriminator trains on randomly cut patches, with generated and Gaussian-blurred real patches as negatives to address few-shot sample imbalance and refine texture detail.The authors describe this patch-based design as computationally efficient and distinguish its sample-sufficiency motivation from PatchGAN's patch-scale structural penalty.Source: §3.2 · pp. 5–6

4.1.1 Datasets.

• The English pre-training set has 32,046 synthetic fonts with 26 glyphs each; the Chinese set has 1,571,940 images across 246 normal fonts and 639 characters.For few-shot testing, the English set uses 35 professional-designed fonts; the Chinese set uses 35 artistic fonts with 7,326 characters each.Source: §4.1.1 · p. 6

4.1.2 Implementation details.

• AGIS-Net uses six convolution and six up-convolution generator layers, three score-map discriminators, and 64×64×3 images, with 32×32×3 patches for the local discriminator.The loss weights remain unchanged across the two training stages.Source: §4.1.2 · p. 6

4.1.3 Competitors.

• The comparison includes BicycleGAN, MS-Pix2Pix, MC-GAN, and TET-GAN; MC-GAN transfers shape and texture but is limited to 26 English capital letters.TET-GAN uses shape outlines as guidance for texture transfer, and the experiments require it to transfer glyph shape as well.Source: §4.1.3 · pp. 6–7
• Evaluation uses IS, FID, SSIM, and pixel accuracy, with higher values preferred for IS, SSIM, and pixel accuracy and lower values for FID.IS measures realism and diversity; FID measures distribution distance, while SSIM and pixel accuracy assess structure and pixel-level agreement.Source: §4.1.4 · p. 7

4.2 Pre-training Results

• During pre-training, BicycleGAN and MS-Pix2Pix struggle with glyph style; MS-Pix2Pix also produces incorrect content, and both fail to converge on the Chinese dataset.The authors therefore omit these methods from the more challenging few-shot comparison.Source: §4.2 · p. 7, Figure 7 · p. 7

4.3 Parameter Studies

• The study conducts experiments to identify an optimal setting for novel glyph synthesis, examining how few-shot reference-set size n and style-input-set size m may affect synthesis performance.Source: §4.3 · p. 7
• On English data, increasing the few-shot set improves contours and reduces fuzzy regions, while too few style inputs (m = 2) produce less satisfactory results.The authors use n = 5 and m = 4 in later English experiments to balance image quality and model size; they caution that IS is not informative for this task.Source: §4.3 · p. 7
• On Chinese data, m = 8 yields results almost the same quality as m = 4, and n = 30 is described as already good enough for practical use.The authors fix n = 30 and m = 4 for subsequent Chinese experiments; the reference sets are randomly selected.Source: §4.3 · p. 7

4.4 Effects of Input Data

• Changing the content-input font among Code New Roman, Courier, Noteworthy, and Marker Felt generally produces insignificant differences for the same character.The authors attribute this to the content encoder learning to ignore style information.Source: §4.4.1 · p. 8, Figure 10 · p. 8

4.4.2 Style few-shot reference set.

• Different style reference sets yield consistent color and texture, but can produce slightly different shapes for fonts whose characters vary in shape style.For a font whose characters mostly share one shape style, the results show no marked difference across reference sets.Source: §4.4.2 · p. 8, Figure 11 · p. 8

4.5 Ablation Studies

• Without skip connections, glyph shape style remains well handled, but texture style such as colors and white lines does not.The figure compares the full model with a version without skip connections.Source: §4.5 · p. 8, Figure 12 · p. 9
• Removing local texture refinement degrades texture details and adds noise, while contextual loss helps preserve shape-outline style.The reported examples include glyphs from both tested fonts.Source: §4.5 · p. 9, Figure 12 · p. 9
• Ablations across all 35 English fonts identify skip connections as the most important component, followed by local texture refinement and contextual loss.The authors associate the latter two with better local texture details and noise-free results, and better shape outlines, respectively.Source: §4.5 · p. 9, Table 3 · p. 9, Figure 12 · p. 9

4.6 Comparison with the State of the Art

• TET-GAN requires binary glyph images as input reference, unlike the task setting in which glyph shape is also synthesized.The comparison uses the same few-shot learning data and fixes content inputs to Code New Roman across the three methods.Source: §4.6 · p. 9

4.6.1 Visual comparison.

• Visual comparisons report that AGIS-Net outperforms existing approaches in transferring both glyph shape and texture styles.The reported results show more complete shape outlines and uniform texture styles than MC-GAN and TET-GAN.Source: §4.6.1 · pp. 9–10, Figure 14 · p. 10
• AGIS-Net performs much better while requiring fewer parameters than other state-of-the-art methods in the English comparisons.Table 4 compares AGIS-Net, TET-GAN, and MC-GAN on English glyph images; Table 5 compares model sizes.Source: §4.6.2 · p. 10, Table 4 · p. 10, Table 5 · p. 10

4.6.2 Quantitative evaluation.

• The Chinese evaluation compares AGIS-Net with TET-GAN on characters outside the few-shot reference set.Table 6 identifies the comparison as quantitative and on the Chinese glyph image dataset.Source: Figure 15 · p. 10, Table 6 · p. 10

4.7 Application on Chinese Glyph Image Synthesis

• The section uses the Chinese dataset to verify AGIS-Net’s extendibility, with style and content inputs described as flexibly controllable.Source: §4.7 · p. 10
• The experiments train and test different methods on the Chinese dataset, with results presented in Figure 15 and Table 6.Source: §4.7 · p. 10
• AGIS-Net transfers Chinese glyphs’ overall and detailed font styles while preserving character content, whereas TET-GAN transfers neither style satisfactorily.The reported comparison is with TET-GAN because MC-GAN handles only 26 Latin capital letters.Source: §4.7 · p. 11

4.8.1 Synthesizing unexplored glyph images.

• For unseen English lowercase letters, results range from satisfactory or reasonable to style-dependent, with ‘a’ and ‘m’ failing.The model tends to produce ‘o’-like results for ‘a’; ‘m’ has a distinctive three-vertical-line structure.Source: §4.8.1 · p. 11
• For unseen Chinese characters, synthesis quality is reported as comparable to explored characters across pre-training and fine-tuning styles.The pre-training set used 500 characters; fine-tuning styles used 30 stylized glyph images.Source: §4.8.1 · p. 11

4.8.2 Cross-language evaluation.

• Chinese-trained AGIS-Net produces impressive Japanese and Korean glyph synthesis results using Omniglot characters as content inputs.The reported conclusion is that the experiments demonstrate generalization ability and content-style disentanglement for glyph images.Source: §4.8.2 · p. 11

• Chinese color inconsistency may reflect a training local minimum that could be addressed by tuning hyperparameters; two English fonts with unique shapes also perform poorly.Source: §4.9 · p. 11

• The comparison removes the texture decoder and Dtex to assess the shape decoder, using selected English pre-training test fonts for few-shot learning.Source: §4.10 · p. 11
• AGIS-Net clearly outperforms EMD on all selected fonts in shape-style transfer.Source: §4.10 · p. 12
• Fonts whose characters mostly have unique shapes or local details remain difficult for the proposed model and other existing approaches.Source: §4.10 · p. 12

5 CONCLUSION

• AGIS-Net synthesizes same-style glyph images for other characters from a small number of training samples.The conclusion describes it as a one-stage few-shot learning model.Source: §5 · p. 12
• Experiments on two publicly available datasets show high-quality bitmap generation that maintains character content and style consistency.Source: §5 · p. 12
• The paper builds a large-scale Chinese glyph image dataset for performance evaluation.Source: §5 · p. 12
• The generated bitmap images cannot directly serve as perfectly scalable vector glyphs, leaving large-scale artistic vector-font synthesis as future work.Source: §5 · p. 12