PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepWriter: A Multi-Stream Deep CNN for Text-independent Writer Identification

Linjie Xing, Yu Qiao

arXiv:1606.06472v2cs.CV

TL;DR

Text-independent writer identification must recognize personal writing style despite arbitrary text content and large within-writer image variation. DeepWriter is a multi-stream CNN trained with softmax identification loss on local handwritten patches, alongside data augmentation and patch scanning for variable-length images. DeepWriter reports 99.01% on 301 writers and 97.3% on 657 writers at English sentence level, and 93.85% on 300 writers at Chinese character level, outperforming previous state-of-the-art methods.

  • Problem

    Text-independent writer identification must recognize personal writing style despite arbitrary text content and large within-writer image variation.

    Source: §I · p. 1

  • Method

    DeepWriter is a multi-stream CNN trained with softmax identification loss on local handwritten patches, alongside data augmentation and patch scanning for variable-length images.

    Source: §I · pp. 1–2, A. Multi-Stream · p. 2

  • Results

    DeepWriter reports 99.01% on 301 writers and 97.3% on 657 writers at English sentence level, and 93.85% on 300 writers at Chinese character level, outperforming previous state-of-the-art methods.

    Source: §I · p. 2, B. Experimental Results · pp. 4–5

  • Takeaways & Limitations

    DeepWriter achieves 98.01% identification accuracy with three adjacent image patches, usually covering 3 to 4 English alphabets.

    Source: B. Experimental Results · p. 4

Abstract

from arXiv · show

Text-independent writer identification is challenging due to the huge variation of written contents and the ambiguous written styles of different writers. This paper proposes DeepWriter, a deep multi-stream CNN to learn deep powerful representation for recognizing writers. DeepWriter takes local handwritten patches as input and is trained with softmax classification loss. The main contributions are: 1) we design and optimize multi-stream structure for writer identification task; 2) we introduce data augmentation learning to enhance the performance of DeepWriter; 3) we introduce a patch scanning strategy to handle text image with different lengths. In addition, we find that different languages such as English and Chinese may share common features for writer identification, and joint training can yield better performance. Experimental results on IAM and HWDB datasets show that our models achieve high identification accuracy: 99.01% on 301 writers and 97.03% on 657 writers with one English sentence input, 93.85% on 300 writers with one Chinese character input, which outperform previous methods with a large margin. Moreover, our models obtain accuracy of 98.01% on 301 writers with only 4 English alphabets as input.

I. INTRODUCTION

Text-independent writer identification must recognize writing style despite arbitrary content and large within-writer variation. DeepWriter uses multiple local image regions, with reported high accuracy across English and Chinese datasets.

I. INTRODUCTION

  • Text-independent identification accepts arbitrary text but is challenging because content differences dominate image variation and obscure writers’ personal writing styles.The authors argue that handcrafted local-shape and gradient features may capture both content and style, limiting performance.Source: §I · p. 1
  • DeepWriter is a multi-stream CNN that learns writer-sensitive features from multiple local regions using softmax identification loss.The stated contributions also include data augmentation, patch scanning for variable-length images, and cross-language feature sharing through pretraining.Source: §I · pp. 1–2
  • DeepWriter reports 99.01% on 301 writers and 97.3% on 657 writers at English sentence level, and 93.85% on 300 writers at Chinese character level.The authors report that these results outperform previous state-of-the-art methods.Source: §I · p. 2

II. RELATED WORKS

Prior work spans handcrafted features for offline identification and CNNs for online identification. The paper positions offline text-independent identification as a more general and difficult setting.

II. RELATED WORKS

  • Writer verification is a one-to-one same-writer decision, while identification searches a known-authorship database and performs multiclass classification.Prior results cited by the authors found lower error rates for verification than identification given the same character count and identical features.Source: §II · p. 2
  • Earlier offline methods combined handcrafted features with distance or classifier-based matching, while the authors argue CNNs can learn task-appropriate features automatically.Bulace et al. reported 89% on 650 IAM writers at page level; Jain et al. reported 93.3% on 300 IAM writers at page level.Source: §II · p. 2
  • CNN-based method [3] addressed online text-independent identification, whereas this paper targets offline identification using scanned grayscale images.The paper characterizes its offline setting as more general and difficult than online identification.Source: §II · p. 2

III. DEEPWRITER

The section introduces DeepWriter’s design and outlines its input preprocessing and training and testing procedures.

III. DEEPWRITER

  • The section first introduces the multi-stream design and preprocessing for variable-length inputs, then describes training and testing implementation.Source: §III · p. 2
  • The section’s stated scope includes implementation details for both training and testing.Source: §III · p. 2
  • The section organizes its discussion around network design and handling handwritten images of varying lengths.Source: §III · p. 2

A. Multi-Stream

DeepWriter processes adjacent handwritten patches to retain some spatial relationship without modeling every patch relationship. The paired outputs are summed, and the design uses the same number of parameters as Half DeepWriter.

A. Multi-Stream

  • DeepWriter uses adjacent 113×113 image patches and sums their FC7 output vectors, preserving local spatial relationships without modeling all patch relationships.The authors describe this as a trade-off between ignoring spatial relationships and the expense of retaining all of them.Source: A. Multi-Stream · pp. 2–3
  • The network diagram identifies convolutional, max-pooling, fully connected, sum, and softmax layers; convolutional and fully connected layers are followed by ReLU.FC6 and FC7 are followed by dropout with ratio 0.5.Source: Figure 4 · p. 3
  • DeepWriter has the same number of model parameters as Half DeepWriter, so the authors state it does not increase overfitting risk or required training-data size.Source: A. Multi-Stream · p. 3

B. Patch Scanning Strategy

The patch-scanning strategy resizes handwritten images while preserving aspect ratio, crops fixed-size patches, and uniformly samples patches for testing.

B. Patch Scanning Strategy

C. Kernel Size

DeepWriter adjusts early convolutional layers to retain image details and uses fewer fully connected neurons, selecting 1024 neurons for FC6 and FC7.

C. Kernel Size

  • Smaller Conv1 and Conv2 kernels and strides were chosen after AlexNet accuracy degraded on 131 × 131 patches, to handle more image details.The adjustment also decreases parameters and, according to the paper, the risk of overfitting.Source: C. Kernel Size · p. 3

D. Neuron Number

  • FC6 and FC7 use fewer neurons than AlexNet, with 1024 selected for each layer after validation-set comparisons.The paper motivates an appropriate neuron count by the smaller training set and number of classes relative to ILSVRC.Source: D. Neuron Number · pp. 3–4
  • The neuron-count experiment varied Half DeepWriter’s FC6 and FC7 sizes on IAM English patches from 301 writers.Source: D. Neuron Number · p. 4, Table 3 · p. 4

E. Feature Sharing

The paper reports shared writer-identification features across languages and describes cross-dataset pretraining and fine-tuning alongside aspect-ratio-preserving patch augmentation.

E. Feature Sharing

  • The authors observe common writer-identification features across languages and compare joint training with training without it.They fine-tune DeepWriter on IAM from Half DeepWriter pretrained on HWDB1.1, then fine-tune Half DeepWriter on HWDB1.1 from that DeepWriter model.Source: E. Feature Sharing · p. 4

F. Training Details

  • Training augmentation randomly crops 113 × 113 patches after resizing while preserving the original aspect ratio.The paper states that distorting the input image seriously degrades identification accuracy.Source: F. Training Details · p. 4
  • Half DeepWriter was trained on HWDB1.1, DeepWriter was fine-tuned on IAM, and Half DeepWriter was then fine-tuned from that model.The reported schedules use batch size 256; training stops after 400K iterations for HWDB1.1 and 40K for IAM.Source: F. Training Details · p. 4

G. Testing Details

Testing scans each handwritten image into patches, obtains writer score vectors for them, averages those vectors, and returns the highest-scoring writer.

G. Testing Details

  • Testing first scans the image to generate patches, then obtains a score vector for each patch or patch pair using DeepWriter or Half DeepWriter.Source: G. Testing Details · p. 4
  • The final writer score is the average of the score vectors across all image patches or patch pairs.The paper treats each output score vector as a probability distribution over writers.Source: G. Testing Details · p. 4
  • The system returns the writer with the highest final score.Source: G. Testing Details · p. 4

IV. EXPERIMENTS

The experiments evaluate DeepWriter on English and Chinese handwriting datasets and report high identification accuracy, including with very little input text. The paper also compares its approach with published methods.

A. Data sets

  • IAM contains unconstrained English handwriting from 657 writers, while HWDB1.1 contains Chinese handwriting from 300 writers; both datasets are divided into training, validation, and testing sets at a 4:1:1 ratio.IAM evaluation uses 301 writers with more than one scanned page; HWDB1.1 contains 1,172,907 character images.Source: A. Data sets · p. 4

B. Experimental Results

  • 99.01% on 301 writers and 97.3% on 657 writers (English sentence level), and 93.85% on 300 writers (one Chinese character); DeepWriter outperforms prior methods by a large margin.With three adjacent patches, usually covering 3 to 4 English alphabets, accuracy is 98.01%; with two patches, usually covering 2 to 3 alphabets, it is 96.92%.Source: B. Experimental Results · pp. 4–5
  • The testing pipeline uses two streams that share the same parameters.Source: Figure 6 · p. 5

V. CONCLUSION AND FUTURE WORK

The conclusion presents DeepWriter as a data-driven CNN for identifying writers from scanned handwriting, including with very short inputs. It highlights patch-based features, multi-stream structure, and data augmentation, and identifies writer verification and joint identification-verification learning as future work.

  • DeepWriter extracts discriminative features from handwritten image patches for text-independent identification of writers from scanned handwriting.Source: §V · p. 5
  • The paper investigates network structure and data augmentation, introducing multi-stream processing to use spatial relationships between handwritten patches.Source: §V · p. 5
  • The reported high accuracy includes settings with one Chinese character or four English alphabets as input.Source: §V · p. 5
  • Future work will investigate offline text-independent writer verification and multitask learning of identification and verification.Source: §V · p. 5

Related papers

Loading 1606.06472v2…