PaperlayerView exact revision on arXiv

Source-linked AI summary

Generative Adversarial Network for Handwritten Text

Bo Ji, Tianyi Chen

arXiv:1907.11845v3cs.LGcs.CV

TL;DR

Handwritten digital-ink generation is less explored than handwriting recognition, and existing RNN generators can produce text that is not as natural as human writing. HWGANs combine a stroke-feature discriminator with a handwritten generator; the authors report more realistic, spatially uniform, neat, and sometimes more diverse samples than Alex’s Generator.

  • Problem

    Handwritten digital-ink generation is relatively less explored, and existing RNN generators can produce samples that are not as natural as ordinary human writing.

    Source: §1 · p. 1

  • Method

    HWGANs combine a Path Signature Feature-based CNN-LSTM and feedforward discriminator with a handwritten generator, including text-conditioned synthesis.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    HWGANs are reported as more realistic than Alex’s Generator, with more spatially uniform strokes and handwriting that is neater and sometimes more diverse in style.

    Source: §1 · p. 2, §5 · p. 7

  • Takeaways & Limitations

    The authors conclude that adversarial training is beneficial for generating more realistic handwritten text data.

    Source: §5 · p. 7

Abstract

from arXiv · show

Generative adversarial networks (GANs) have proven hugely successful in variety of applications of image processing. However, generative adversarial networks for handwriting is relatively rare somehow because of difficulty of handling sequential handwriting data by Convolutional Neural Network (CNN). In this paper, we propose a handwriting generative adversarial network framework (HWGANs) for synthesizing handwritten stroke data. The main features of the new framework include: (i) A discriminator consists of an integrated CNN-Long-Short-Term- Memory (LSTM) based feature extraction with Path Signature Features (PSF) as input and a Feedforward Neural Network (FNN) based binary classifier; (ii) A recurrent latent variable model as generator for synthesizing sequential handwritten data. The numerical experiments show the effectivity of the new model. Moreover, comparing with sole handwriting generator, the HWGANs synthesize more natural and realistic handwritten text.

1 Introduction

Handwritten digital-ink generation remains less explored, and existing RNN generators can produce text that is not as natural as human writing. HWGANs pair a stroke-feature discriminator with a handwritten generator and report more realistic text than Alex’s Generator.

1 Introduction

  • Handwritten digital ink is less exploited than recognition, and existing RNN generators [1] [2] [3] can produce text that is less natural than ordinary human writing.Source: §1 · p. 1
  • GANs are less applied to sequential generation, motivating their use to seek more realistic digital-ink handwriting than a handwritten generator alone.Source: §1 · p. 1
  • The discriminator extracts Path Signature Features from strokes with a compact CNN-LSTM and uses a feedforward network to distinguish genuine from forged handwriting.The extracted features encode handwritten strokes before classification.Source: §1 · p. 2
  • The proposed adversarial architecture combines the designed discriminator with Alex’s Generator [1].Source: §1 · p. 2
  • HWGANs produce more realistic English handwriting than Alex’s Generator, with more spatially uniform strokes and text that is neater and sometimes more stylistically diverse.The authors report these as general comparisons, not universal handwriting-quality metrics.Source: §1 · p. 2

2 Related Work

Prior work includes image-based handwriting generation, a variational recurrent generator, and an RNN-based digital-ink generator. Their described capabilities and limitations differ, especially in whether they directly produce stroke data or condition generation on text.

2 Related Work

  • The 2019 handwritten-text image GAN [7] uses a word embedding, a generator, and an auxiliary text-recognition network, but needs Ink Grab to convert images into digital strokes.Source: §2 · p. 2
  • VRNN [8] combines a recurrent variational autoencoder with latent variables in an RNN hidden state, but its synthesized handwriting is reported as not realistic enough.The work also demonstrated generation of speech and handwritten data.Source: §2 · p. 2
  • Alex Grave’s RNN generator [1] predicts each next stroke point from the sampled path prefix and includes a synthesizer that generates handwriting for given text.Source: §2 · p. 2

3 Proposed Adversarial Approach

HWGANs use a discriminator to distinguish realistic handwritten text from forgeries and a generator to produce handwriting. The discriminator encodes stroke geometry and order, while the generator supports prediction and text-conditioned synthesis.

3 Proposed Adversarial Approach

  • The framework pairs a discriminator that distinguishes realistic handwriting from forgeries with a generator designed to produce handwriting.Source: §3 · pp. 2–3

3.1 Discriminator

  • Path Signature Features encode stroke geometry and order into seven maps, which form the input tensor for a CNN-LSTM classifier topped by a feedforward binary classifier.The CNN output has variable columns, which are processed sequentially by the LSTM; the discriminator uses binary cross-entropy loss.Source: §3.1 · p. 3

3.2 Generator

  • The generators are stacked LSTMs that predict next-point distributions, update parameters by maximizing sequence log-likelihood, and use discriminator feedback during adversarial training.Both prediction and synthesis generators process generated strokes as Path Signature Features for the discriminator.Source: §3.2 · pp. 3–4
  • The synthesis generator Gs receives stroke points and user-specified text, represented as a one-hot vector and fed to LSTM layers with window parameters.Source: §3.2 · p. 4

3.1 Discriminator

  • Table I gives the CNN-LSTM architecture configuration and defines its input, kernel, and output shape conventions.Source: Table I · p. 5

4 Experiments

Experiments compare HWGANs with Alex’s Generator across prediction and synthesis, including tests at varying sampling biases and on different prescribed sentences. Reported outcomes favor HWGANs on several qualitative properties, while the prediction comparison is described as competitive rather than decisively better.

4.1 Experimental Settings

  • Training used IAM’s selected 54-character subset, 20 Gaussian mixture components for next-point coordinates, normalized PSF features, label smoothing, and real and generated discriminator samples.Source: §4.1 · pp. 5–6

4.2 Numerical Results

  • HWGANs’ prediction samples were competitive, while synthesis samples were described as more realistic, natural, and neat than Alex’s Generator.The authors caution that handwriting-quality judgments lack a universal standard and do not claim a winner in the prediction comparison.Source: §4.2 · p. 7
  • At sampling biases 4, 5, 7, and 10, HWGANs’ samples were reported as more spatially uniform and neat, with possibly more diverse styles.The authors note that sampling bias has a large impact on results.Source: §4.2 · p. 7
  • With sampling bias fixed at 5.0, the reported advantages remained consistent across different prescribed sentences.Source: §4.2 · p. 7

5 conclusions

The conclusion presents HWGANs as a digital-ink handwriting generator pairing PSF/CNN-LSTM discrimination with Alex Grave-style generators. It reports qualitative advantages over Alex’s Generator in spatial uniformness, neatness, and font-style diversity.

  • HWGANs pair a PSF feature extractor and CNN-LSTM binary classifier with Alex Grave-style generators, and are reported to improve spatial uniformness, neatness, and font-style diversity over Alex’s Generator.Source: §5 · p. 7

Related papers

Loading 1907.11845v3…