Source-linked AI summary
Learning Features for Offline Handwritten Signature Verification using Deep Convolutional Neural Networks
Luiz G. Hafemann, Robert Sabourin, Luiz S. Oliveira
TL;DR
Offline signature verification must distinguish genuine signatures from skilled forgeries despite lost writing dynamics and difficult feature design. The paper learns writer-independent representations with convolutional neural networks, using forgery knowledge during feature learning, and reports improved performance across datasets, including 1.72% EER on GPDS-160 versus 6.97% in the literature.
Problem
Offline verification loses dynamic writing information, while designing features that distinguish genuine signatures from skilled forgeries remains difficult.
Method
The paper learns writer-independent signature-image representations with convolutional neural networks and formulates feature learning to use forgery knowledge from selected users.
Results
1.72% Equal Error Rate was obtained on GPDS-160 versus 6.97% reported in the literature, while features surpassed state-of-the-art performance on users from other datasets.
Takeaways & Limitations
The learned features generalized beyond GPDS, surpassing state-of-the-art performance on other datasets without dataset-specific fine-tuning.
Takeaways & Limitations
The model learned with GPDS forgeries did not perform better in all cases, suggesting that forgery characteristics may differ across datasets.
Abstract
from arXiv · showhide
Verifying the identity of a person using handwritten signatures is challenging in the presence of skilled forgeries, where a forger has access to a person's signature and deliberately attempt to imitate it. In offline (static) signature verification, the dynamic information of the signature writing process is lost, and it is difficult to design good feature extractors that can distinguish genuine signatures and skilled forgeries. This reflects in a relatively poor performance, with verification errors around 7% in the best systems in the literature. To address both the difficulty of obtaining good features, as well as improve system performance, we propose learning the representations from signature images, in a Writer-Independent format, using Convolutional Neural Networks. In particular, we propose a novel formulation of the problem that includes knowledge of skilled forgeries from a subset of users in the feature learning process, that aims to capture visual cues that distinguish genuine signatures and forgeries regardless of the user. Extensive experiments were conducted on four datasets: GPDS, MCYT, CEDAR and Brazilian PUC-PR datasets. On GPDS-160, we obtained a large improvement in state-of-the-art performance, achieving 1.72% Equal Error Rate, compared to 6.97% in the literature. We also verified that the features generalize beyond the GPDS dataset, surpassing the state-of-the-art performance in the other datasets, without requiring the representation to be fine-tuned to each particular dataset.
1. Introduction
Offline signature verification loses writing-process dynamics, making discriminative feature design and skilled-forgery detection difficult. The paper learns Writer-Independent representations from signature images with CNNs, using skilled-forgery knowledge to improve performance and generalization across datasets.
- Problem: Offline signatures are static images, so dynamic information such as pen position and velocity over time is lost.This makes verification particularly challenging when distinguishing genuine signatures from skilled forgeries.
- Problem: No feature extractor has emerged as particularly suitable, while genuine signatures can vary substantially and skilled forgeries may closely resemble them.These variations challenge grid-, character-, and directional-based descriptors.
- Method: The proposed framework learns signature representations directly from images using convolutional neural networks in a Writer-Independent format.Its formulation uses multi-task learning to jointly discriminate users and genuine signatures from skilled forgeries.
- Method: Skilled forgeries from a subset of users guide feature learning toward visual cues that distinguish genuine signatures and forgeries across users.The learned representation is evaluated on users without available skilled forgeries.
- Results: 1.72% Equal Error Rate was achieved on GPDS-160, compared with 6.97% reported in the literature.Features learned on GPDS were also used to train classifiers on MCYT, CEDAR, and Brazilian PUC-PR, surpassing state-of-the-art performance there.
2. Related works
Offline signature verification has been approached with writer-dependent and writer-independent classifiers, while feature learning from raw signature images remains comparatively underexplored. The paper’s related work distinguishes random from skilled forgeries and motivates learning representations that address skilled-forgery discrimination.
- Offline Signature Verification: Writer-dependent systems train one binary classifier per user using genuine signatures as positives and other users’ signatures as random-forgery negatives.This approach performs well but becomes more complex as users are enrolled.
- Offline Signature Verification: Writer-independent systems use a single model for all users by classifying dissimilarity vectors between query and template signature features.They reduce complexity but often perform worse than writer-dependent systems, with the best standard-benchmark results typically coming from writer-dependent systems.
- Feature Extraction: Prior work has explored geometric, graphology-inspired, directional, interest-point, and texture descriptors, commonly extracted locally from signature-image grids.Examples include HOG, D-PDF, SIFT, LBP, and GLCM.
- Representation Learning: Feature learning for offline signature verification has been less widely explored, and earlier studies either did not test learned features for classification or did not consider skilled forgeries.The paper instead learns representations directly from pixel intensities rather than relying on handcrafted features.
- Representation Learning: The paper differs from related approaches by learning representations from raw signature pixels and using a Development dataset for feature learning rather than feature selection.CNNs learn shared filters across image positions and can transfer representations from larger datasets to tasks with fewer available samples.
3. Feature learning for Signature Verification
The paper learns signature representations across writers with CNNs, then trains user-specific classifiers in the learned space. It extends this framework by incorporating skilled-forgery information to learn cues that separate genuine signatures from forgeries.
- Writer-Independent feature learning: The two-phase system learns a CNN representation, projects signatures into that space, and trains a binary classifier for each user.The classifiers model individual-user characteristics using the learned representation.
- Writer-Independent feature learning: Writer-Independent feature learning uses signatures from many users to learn intrinsic properties that can generalize to new users.Feature learning and Writer-Dependent classification use disjoint user sets to test generalization.
- CNN representations: CNNs preserve detailed signature-image information while extracting local features and combining them nonlinearly across layers.The architecture is motivated by larger input sizes and the need to retain discriminative pen-stroke details.
- Learning from skilled forgeries: Training only to distinguish users provides no direct pressure for the learned features to distinguish skilled forgeries.The paper therefore motivates incorporating forgery knowledge into feature learning.
- Learning from skilled forgeries: The proposed forgery-aware formulations either assign separate genuine and forgery classes to each user or use multi-task learning.The multi-task formulation combines user classification with forgery classification, using a hyperparameter to trade off the objectives.
- Learning from skilled forgeries: The multi-task forgery output predicts whether a signature is forged and encourages visual cues that distinguish genuine signatures from forgeries regardless of user.Training can use skilled forgeries from a subset of users rather than requiring them for every enrolled user.
4. Experimental Protocol
The experiments use separate development, validation, and exploitation partitions to learn CNN representations, select models, and evaluate Writer-Dependent classifiers across four signature datasets.
- Datasets: Experiments covered GPDS-960, MCYT-75, CEDAR, and Brazilian PUC-PR signature datasets.
- Exploitation protocol: Final classifiers were trained and tested on exploitation-set partitions using Support Vector Machines and genuine signatures from enrolled users.
- Data partitioning: GPDS development data trained CNNs on 531 users, with disjoint learning and validation samples split 90% and 10%.
- Validation: The validation set Vv contained 50 users and supported Writer-Dependent classifier evaluation and all CNN architecture and hyperparameter choices.
- Feature learning: Forgery-aware feature learning varied λ from 0 to 1, with additional tests at 0.95, 0.99, and 0.999 after better results near the upper boundary.
- Evaluation: Performance used FRR, random- and skilled-forgery FAR, EER under user-specific or global thresholds, and mean per-user AUC.
5. Results and Discussion
The results show that forgery-aware CNN feature learning improves validation behavior and achieves strong verification performance across GPDS, MCYT, CEDAR, and Brazilian PUC-PR.
- Representation learning: Forgery-aware learning performed better near λ=1, suggesting that user-independent forgery distinctions were more relevant than user classification.
- Validation results: 3.13% EER with an RBF SVM and 3.91% with a linear SVM were obtained using the separate forgery-neuron formulation.
- Validation results: The separate forgery-neuron formulation outperformed treating forgeries as separate classes, which produced 5.61% EER with a linear SVM.
- Validation results: 99.23% classification performance was achieved on GPDS validation set Vc for distinguishing 531 users.
- Thresholding: User-specific thresholds outperformed a global threshold on GPDS-160, with 1.72% versus 3.61% EER.
- Exploitation results: 1.72% EER on GPDS-160 compared with 6.97% state of the art was achieved using 12 samples per user and a single classifier.
- Cross-dataset results: State-of-the-art performance was reported across all datasets, including 2.87% EER on MCYT and 4.63% EER on CEDAR.
- Sample efficiency: GPDS achieved 5.74% EER with one sample per user, surpassing a state-of-the-art system using 12 samples.
6. Conclusion
The paper learns writer-independent representations for offline signature verification and shows strong performance, including with few genuine samples and across datasets. It also identifies a limitation: forgery characteristics may differ between datasets, limiting consistent gains from forgery-informed training.
- Writer-independent learned features improve offline signature verification compared with hand-engineered features.
- 1.72% EER was obtained on GPDS-160, compared with 6.97% reported in the literature, using forgery data from a subset of users.
- 2.41% EER was obtained on GPDS with five genuine samples per user, illustrating strong performance when few samples are available.
- The learned features generalized to unseen users within GPDS and to users in MCYT, CEDAR, and Brazilian PUC-PR, surpassing state-of-the-art performance on all three other datasets.
- The model learned with GPDS forgeries did not perform better in all cases, suggesting that forgery characteristics may differ across datasets.