Source-linked AI summary
GTC: Guided Training of CTC Towards Efficient and Accurate Scene Text Recognition
Wenyang Hu, Xiaocong Cai, Jun Hou, Shuai Yi, Zhiping Lin
TL;DR
Scene text recognition requires balancing CTC’s fast inference against attention methods’ stronger accuracy. GTC trains CTC with attentional guidance and adds GCN-based feature correlations, achieving strong benchmark performance while preserving speed.
Problem
CTC-based recognition is efficient but less effective than attention-based recognition because its training can mislead feature alignments and representations.
Method
GTC uses an attention decoder as training guidance for a CTC model and adds a GCN to capture correlations among feature-sequence slices.
Results
6 times shorter inference time than attention-based methods and new state-of-the-art performance on most regular and irregular scene text datasets are reported.
Takeaways & Limitations
The proposed framework improves CTC robustness and accuracy while retaining efficient inference across regular and irregular scene text recognition benchmarks.
Takeaways & Limitations
The method assumes neighboring CTC time steps contain supplementary features and correlations within the feature sequence.
Abstract
from arXiv · showhide
Connectionist Temporal Classification (CTC) and attention mechanism are two main approaches used in recent scene text recognition works. Compared with attention-based methods, CTC decoder has a much shorter inference time, yet a lower accuracy. To design an efficient and effective model, we propose the guided training of CTC (GTC), where CTC model learns a better alignment and feature representations from a more powerful attentional guidance. With the benefit of guided training, CTC model achieves robust and accurate prediction for both regular and irregular scene text while maintaining a fast inference speed. Moreover, to further leverage the potential of CTC decoder, a graph convolutional network (GCN) is proposed to learn the local correlations of extracted features. Extensive experiments on standard benchmarks demonstrate that our end-to-end model achieves a new state-of-the-art for regular and irregular scene text recognition and needs 6 times shorter inference time than attentionbased methods.
Introduction
Scene text recognition needs a decoder that combines attention-level accuracy with CTC-level efficiency. GTC addresses this by training CTC from attentional guidance and adding GCN-based feature correlations.
- Motivation: CTC methods enable fast parallel decoding but generally learn weaker alignments and feature representations than attention-based methods.This creates an efficiency–accuracy trade-off in scene text recognition.
- GTC: GTC trains a CTC model using an attention decoder as guidance, while retaining the CTC decoder for both training and evaluation.The guidance improves the encoder representations and is not used during inference.
- GCN enhancement: A GCN captures correlations among neighboring sequence slices and merges supplementary features belonging to the same label.This targets errors caused when repeated CTC time steps contain only partial character evidence.
- GTC: GTC is designed as a flexible optimization method that can use different guidance networks to improve CTC-based recognition.The paper presents it as a general method rather than a single fixed architecture.
- Contributions: The paper reports GCN-based scene text recognition as a first attempt that improves CTC decoder accuracy and robustness.The contribution is supported by extensive experiments.
- Results: 6 times shorter inference time and new state-of-the-art performance on most regular and irregular datasets are reported for the proposed method.The result is presented as improving efficiency and effectiveness relative to other works.
Related Work
Prior scene text recognition methods include segmentation, classification, attention, and CTC-based approaches, each with practical limitations. This work combines guided CTC optimization with GCN-based sequence correlations to target the accuracy–efficiency gap.
- Earlier approaches: Early segmentation-based methods struggle with complicated backgrounds and varied text alignments, while dictionary-based word classification is constrained by dictionary size.These limitations motivate sequence-based recognition approaches.
- CTC and attention: CTC-based methods provide fast inference, whereas attention-based methods generally achieve higher accuracy on irregular text but decode more slowly.STN+CTC remains fast, but the CTC decoder limits performance.
- Proposed direction: The proposed approach differs by using guided training for better image representations and GCN layers for spatial and contextual sequence correlations.The paper characterizes this as an early use of both techniques in scene text recognition.
- Proposed direction: The paper attributes poor direct CTC–attention combinations to degraded feature-representation learning and uses guidance to improve CTC performance.This distinguishes guided optimization from simply combining the two decoders.
Methodology
GTC trains a fast CTC decoder with an attentional guidance branch that supplies better alignments and feature representations, while a GCN models local feature correlations. The framework combines spatial rectification, CNN feature extraction, attention guidance, and GCN-enhanced CTC decoding.
- Guided CTC Training: GTC uses an attentional decoder during training to guide the CTC model toward better feature representations and alignments.The attentional guidance is not used during evaluation, preserving efficient CTC inference.
- Network Architecture: The framework contains an STN, ResNet backbone, attentional guidance branch, and GCN-powered CTC decoder.The STN normalizes text images, while the ResNet extracts a fixed-height feature sequence with varying length.
- Attentional Guidance: The attention decoder uses a GRU to update hidden states from previous outputs and glimpse vectors computed with attention weights over the feature sequence.During training, the previous predicted character is replaced by the ground-truth character.
- Feature Extraction: The feature extractor configuration uses residual-block stride 1 and an overall down-sampling ratio of W: 1/4 and H: 1/16.The configuration specifies convolutional, residual, max-pooling, and average-pooling operations.
- GCN+CTC Decoder: The GCN learns feature correlations from pairwise similarity and distance relations before BiLSTM sequence modeling.Similarity is computed from transformed feature slices, while distance relations emphasize neighboring features.
- GCN+CTC Decoder: The GCN+CTC decoder converts graph-enhanced features into classification logits and trains them with CTC loss.The graph output is passed through a BiLSTM before classification.
Experiments
Experiments evaluate GTC on regular and irregular scene-text benchmarks under lexicon-free conditions, comparing recognition accuracy, inference speed, and guided-training variants. GTC improves CTC performance while retaining fast decoding, and its GCN module further supports sequence modeling for irregular text.
- Evaluation setup: Experiments cover regular and irregular scene-text datasets, including public benchmarks evaluated in a lexicon-free setting.The evaluation includes seven public datasets; some comparisons also use rotation strategies or alternative guidance.
- Recognition accuracy: Table 2 reports text-line recognition accuracy across public benchmarks, with the best GTC model leading overall recognition and achieving state-of-the-art on most datasets.The table compares lexicon-free results and marks methods using rotation or character-level annotations.
- Ablation and analysis: The GCN module models a better feature sequence for irregular text while all GTC results use CTC-decoder predictions.The adjacency construction combines similarity and distance information to focus on local similar features.
- Efficiency: 6 times faster inference and the highest recognition rate distinguish GTC from attention-based methods.Measured on a single NVIDIA Titan X with batch size 1, attention decoding takes 54 ms/image while GCN+CTC takes 3.7 ms/image.
- Efficiency: 14+ times faster inference is achieved by the GCN+CTC decoder than the attention decoder.The reported times are 3.7 ms/image for GCN+CTC and 54 ms/image for attention decoding.
- Ablation and analysis: Using CTC to guide attention produces poor feature representations, indicating that effective guidance is necessary for guided training.The comparison reports that CTC is not as effective a guide as attention and that CTC loss harms the training process.
- Ablation and analysis: GTC produces better transformation results than the STN+CTC framework, while attention’s spatial sensitivity makes the transformation module more responsive.This comparison is illustrated through original and rectified text images.
Conclusion
The paper presents GTC as an efficient CTC optimization method and introduces GCN-based local correlation modeling. Experiments on public benchmarks report state-of-the-art performance on most datasets.
- Conclusion: GTC improves robustness and performance in scene text recognition by addressing limitations of CTC.The method is presented as both effective and efficient.
- Conclusion: GCN learns local correlations in feature sequences and further improves CTC performance.The paper describes this as its first use of GCN for local-correlation learning in scene text recognition.
- Conclusion: State-of-the-art performance is achieved on most datasets across the seven public benchmarks used for evaluation.The conclusion summarizes experiments assessing the proposed method’s effectiveness.