PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Write Stylized Chinese Characters by Reading a Handful of Examples

Danyang Sun, Tongzheng Ren, Chongxun Li, Hang Su, Jun Zhu

arXiv:1712.06424v3cs.CVcs.LGstat.ML

TL;DR

Generating stylized Chinese characters remains challenging because of their complex structures and the limited ability of prior approaches to handle new styles from few examples. SA-VAE separates content and style while incorporating character knowledge, and the authors report one-shot and few-shot generalization to unseen styles, including printing and handwritten characters.

  • Problem

    Chinese character generation is underexplored, and existing approaches have difficulty handling complex structures or generating new styles from only a few examples.

    Source: §1 · p. 1

  • Method

    SA-VAE disentangles content and style with intercross pair-wise optimization and incorporates Chinese-character domain knowledge to guide generation.

    Source: §1.1 · p. 2

  • Results

    Experiments demonstrate one-shot and few-shot generalization to unseen styles, with stylized characters in both printing and handwritten styles.

    Source: §4 · p. 7

  • Takeaways & Limitations

    SA-VAE generates stylized Chinese characters by observing only one or a few examples, including examples in styles unseen during training.

    Source: §4 · p. 7

    The formulation assumes that each Chinese character is independently determined by content and style factors.

    Source: §2.1 · p. 2

Abstract

from arXiv · show

Automatically writing stylized Chinese characters is an attractive yet challenging task due to its wide applicabilities. In this paper, we propose a novel framework named Style-Aware Variational Auto-Encoder (SA-VAE) to flexibly generate Chinese characters. Specifically, we propose to capture the different characteristics of a Chinese character by disentangling the latent features into content-related and style-related components. Considering of the complex shapes and structures, we incorporate the structure information as prior knowledge into our framework to guide the generation. Our framework shows a powerful one-shot/low-shot generalization ability by inferring the style component given a character with unseen style. To the best of our knowledge, this is the first attempt to learn to write new-style Chinese characters by observing only one or a few examples. Extensive experiments demonstrate its effectiveness in generating different stylized Chinese characters by fusing the feature vectors corresponding to different contents and styles, which is of significant importance in real-world applications.

1 Introduction

Stylized Chinese character generation is underexplored and difficult because of the characters’ scale and structural complexity, while existing style-transfer approaches require retraining for new styles. The paper proposes SA-VAE to separate content and style, incorporate character structure knowledge, and generate characters in one-shot or few-shot settings.

1 Introduction

  • Character generation is less explored than recognition, and writing stylized Chinese characters from a handful of examples remains challenging for machines despite being feasible for people.The authors frame this as a testbed for machine intelligence and note Chinese characters’ importance across languages.Source: §1 · p. 1
  • Chinese characters’ large dictionary and complex structures challenge generation; existing methods either omit style, rely on online stroke timing, or require clearly parsed components.Earlier style-transfer systems also require retraining from scratch for a new style.Source: §1 · pp. 1–2

1.1 Our Proposal

  • SA-VAE extends VAE with separate style-inference and content-recognition networks, using intercross pair-wise optimization to encourage pure, reliable style embeddings.The authors motivate deep nonlinear function approximation as better suited to complex combinations than bilinear matrix decomposition.Source: §1.1 · p. 2
  • The framework incorporates Chinese-character configuration and radical information as prior knowledge, and learns a style bank from printing and handwritten examples.The paper says this knowledge benefits generation of characters with complex structures and that new styles can be inferred without retraining.Source: §1.1 · p. 2
  • Experiments demonstrate stylized character generation from one or a few examples, including unseen styles, with results for both printing and handwritten styles.The authors describe this as the first attempt to learn new-style Chinese character writing in a one-shot or few-shot scenario.Source: §1.1 · p. 2

2 Methodology

The methodology separates character content from writing style, then recombines those representations to generate stylized characters. It adds Chinese-character structure information and trains with intercross pairs to support inference from a few examples.

2.1 Problem Formulation

  • Assuming content and style are independent, SA-VAE infers style from observed characters and combines it with content codes to generate characters in new styles.The style-bank contains characters across multiple styles and contents.Source: §2.1 · p. 2

2.2 Model Architecture

  • C recognizes content, S infers style, and G decodes both; K encodes character configurations and radicals to avoid one-hot parameter growth across thousands of classes.The encoding reuses shared configuration and radical information.Source: §2.2 · p. 3

2.3 Intercross Pairwise Optimization

  • Intercross training reconstructs a target using same-style, different-content character pairs, encouraging matching style posteriors across characters.The objective minimizes the KL term between style posteriors for paired characters; the authors describe it as a lower bound on log-likelihood.Source: §2.3 · p. 4

2.4 One-shot/Few-shot Character Generation

  • For a new style, one observed character provides its style embedding; with several examples, averaging their embeddings reduces the style estimate's variance by a factor of 1/n.The inferred style can be combined with content codes for characters to generate in that style.Source: §2.4 · p. 5

3 Experiments

The experiments evaluate SA-VAE on one- and few-shot stylized character generation, including generalization to new styles, disentanglement, and the use of character-structure knowledge.

3.2 One-shot/Few-shot Generation Results

  • Figure 5 contrasts generated and ground-truth rows in one- and few-shot settings and demonstrates results in printing and handwritten styles.The dotted-frame characters specify the target style.Source: Figure 5 · p. 5, §3.2 · p. 5

3.1 Data Preparation

  • The dataset contains characters in 200 styles: 160 styles form the training style-bank and 40 are held out for testing, using 3,000 frequent characters per training style.Source: §3.1 · p. 5

3.2 One-shot/Few-shot Generation Results

  • SA-VAE generates new styles from one character; with unseen styles and 10 new-style fine-tuning examples, Rewrite and zi2zi hardly capture the style, while results are close when target styles were trained.Table 1 compares generation results across settings.Source: §3.2 · p. 5, Table 1 · p. 6

3.3 Style Generalization

  • Style interpolation and nearest-neighbor comparisons examine generalization, with the neighbors checking that outputs are not simply the most similar training style.Figure 6 shows interpolation of two styles and the generated character alongside its two nearest training-set neighbors.Source: §3.3 · pp. 5–6, Figure 6 · p. 6

3.4 Evaluation of Disentanglement

  • Without intercross pair-wise optimization, style embeddings retain too much content and produce blurry outputs; classifier tests assess style and content information in the embeddings.The authors report that qualitative and quantitative experiments support the method's disentanglement ability.Source: §3.4 · p. 6, Table 2 · p. 6

3.5 Structure Knowledge of Chinese Characters

  • At 500 unique characters, one-hot parameters explode; one-hot may otherwise converge comparably to the knowledgeable content code, while binary embedding hardly converges meaningfully.The experiments compare the convergence of content-code, one-hot, and binary encodings.Source: Figure 8 · p. 6, §3.5 · p. 6

4 Conclusions

The Style-Aware VAE generates stylized Chinese characters from one or a few examples. It disentangles style and content and shows one-shot or few-shot generalization to new styles.

  • SA-VAE generates stylized Chinese characters after observing only one or a few characters.Source: §4 · p. 7
  • Its intercross pair-wise optimization disentangles style information from content information.Source: §4 · p. 7
  • Experiments report generation results for both printing and handwritten styles, with one-shot or few-shot generalization to new styles.Source: §4 · p. 7

A Generation Results of Symbolic Alphabets

SA-VAE is also applied to digits, English letters, and Japanese kanas. The experiments use one-hot labels for these characters and suggest the framework can generalize beyond Chinese character generation.

A Generation Results of Symbolic Alphabets

Related papers

Loading 1712.06424v3…