Source-linked AI summary
ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard
Xi Liu, Rui Zhang, Yongsheng Zhou, Qianyi Jiang, Qi Song, Nan Li, Kai Zhou, Lei Wang, Dong Wang, Minghui Liao, Mingkun Yang, Xiang Bai, Baoguang Shi, Dimosthenis Karatzas, Shijian Lu, C. V. Jawahar
TL;DR
Chinese scene text reading is challenging because of the language’s large character set and varied signboard layouts and fonts. The paper organizes ICDAR2019-ReCTS around a 25,000-image annotated signboard dataset, four recognition and detection tasks, and multi-GT evaluation for ambiguity. The competition attracted 46 teams and hundreds of submissions.
Problem
Chinese scene text reading is challenging because Chinese has more than 6000 commonly used characters and varied layouts, arrangements, and fonts.
Method
The paper organizes a signboard-focused competition with an annotated dataset, four tasks, and multi-GT evaluation for ambiguous text grouping.
Results
46 valid teams participated and hundreds of valid submissions were received across the competition’s four tasks.
Takeaways & Limitations
ReCTS provides a large-scale benchmark for Chinese signboard text reading spanning character, text-line, detection, and end-to-end recognition.
Abstract
from arXiv · showhide
Chinese scene text reading is one of the most challenging problems in computer vision and has attracted great interest. Different from English text, Chinese has more than 6000 commonly used characters and Chinesecharacters can be arranged in various layouts with numerous fonts. The Chinese signboards in street view are a good choice for Chinese scene text images since they have different backgrounds, fonts and layouts. We organized a competition called ICDAR2019-ReCTS, which mainly focuses on reading Chinese text on signboard. This report presents the final results of the competition. A large-scale dataset of 25,000 annotated signboard images, in which all the text lines and characters are annotated with locations and transcriptions, were released. Four tasks, namely character recognition, text line recognition, text line detection and end-to-end recognition were set up. Besides, considering the Chinese text ambiguity issue, we proposed a multi ground truth (multi-GT) evaluation method to make evaluation fairer. The competition started on March 1, 2019 and ended on April 30, 2019. 262 submissions from 46 teams are received. Most of the participants come from universities, research institutes, and tech companies in China. There are also some participants from the United States, Australia, Singapore, and Korea. 21 teams submit results for Task 1, 23 teams submit results for Task 2, 24 teams submit results for Task 3, and 13 teams submit results for Task 4. The official website for the competition is http://rrc.cvc.uab.es/?ch=12.
I. INTRODUCTION
Chinese scene text reading is challenging because Chinese uses more than 6000 commonly used characters and highly varied layouts and fonts. ICDAR2019-ReCTS addresses this challenge with a large signboard dataset, four tasks, and multi-GT evaluation for text ambiguity.
- Chinese scene text reading is difficult because it involves more than 6000 commonly used characters and varied layouts, arrangements, and fonts.
- Signboards provide diverse Chinese scene text images with varied backgrounds, fonts, and layouts.
- ReCTS releases 25,000 annotated signboard images and establishes character recognition, text line recognition, text line detection, and end-to-end recognition tasks.
- Multi-GT evaluation addresses ambiguity over whether neighboring words should be merged by comparing predictions with multiple ground truths and using the best match.
- The competition ran from March 1 to April 30, 2019, attracting 46 valid teams and hundreds of valid submissions.
II. DATASET AND ANNOTATIONS
ReCTS-25k is a 25,000-image Chinese signboard dataset collected under uncontrolled phone-camera conditions. It provides polygon locations and UTF-8 transcriptions for text lines and characters and supports four challenge tasks.
- ReCTS-25k comprises 25,000 signboard images collected by business merchants using phone cameras under uncontrolled conditions.
- The dataset focuses on Chinese signboard text whose layouts and character arrangements are complex for aesthetic or emphasis-related reasons.
- All text lines and characters are manually annotated with four-vertex polygon locations and UTF-8 encoded transcriptions.
- ReCTS includes 20,000 training images and 5,000 test images, with difficult flags for utterly obscure or small text.
- The challenge evaluates character recognition, text line recognition, text line detection, and end-to-end recognition for varied layouts, fonts, and orientations.
A. Task 1 – Character Recognition
Task 1 evaluates recognition of cropped Chinese character images from signboards, where characters appear in diverse fonts. Participants submit character predictions for all test images and are ranked by recognition accuracy.
- Task 1 recognizes characters in cropped signboard character images containing diverse fonts.
- Participants submit a text file containing character results for all test images.
- Recognition accuracy is calculated from the number of correctly predicted characters divided by the total number of test characters.
B. Task 2 – Text Line Recognition
Task 2 recognizes cropped text lines, including perspective-distorted and arbitrarily arranged lines, using normalized edit distance. Task 3 detects text-line polygons, while multi-GT evaluation accommodates ambiguity in text grouping.
- B. Task 2 – Text Line Recognition: Task 2 provides cropped text-line images and polygon coordinates, including perspective and arbitrarily arranged text lines, for recognition.
- B. Task 2 – Text Line Recognition: Task 2 uses average normalized edit distance between predicted text lines and ground truths as its evaluation metric.
- C. Task 3 – Text Line Detection: Task 3 localizes text lines in full signboard images by submitting four-vertex polygons in clockwise order.
- C. Task 3 – Text Line Detection: Multi-GT evaluation treats alternative text grouping decisions as correct by comparing predictions with all provided ground truths and selecting the best match.
- C. Task 3 – Text Line Detection: Task 3 reports Precision, Recall, and F-score at IoU thresholds 0.5 and 0.7, with F-score at IoU=0.5 determining final ranking.
D. Task 4 – End-to-End Recognition
Task 4 evaluates end-to-end recognition by requiring systems to localize and transcribe every text instance in full signboard images. Evaluation matches detections to ground-truth polygons using an IoU threshold, then computes edit distances for matched pairs.
- Task 4 requires localizing and recognizing every text instance in each full signboard image.
- Participants submit each recognized text line with its four-vertex polygon location and UTF-8 transcription.
- Detections are matched to the highest-IoU ground-truth polygon, or to None when no IoU exceeds 0.5.
- Duplicate detections matched to one ground truth are reduced to the single detection with maximum IoU before edit distances are calculated.
IV. ORGANIZATION
The competition ran from March 1 through April 30, 2019 and attracted 46 valid teams across several countries. Teams submitted through the RRC website, with participation and ranking reported separately for each task.
- The competition opened on March 1, 2019 and closed at 11:59 PM PST on April 30 after staged dataset and test-set releases.
- 46 valid teams participated, mainly from Chinese universities, research institutes, and technology companies, with additional teams from four other countries.
- Teams could submit at most five results, and the best result among those submissions became the final result.
- The evaluation script ranked Task 1 by accuracy, Task 2 by normalized edit distance, and Task 3 by F-score.
- Task 4’s top-five results were ranked by normalized edit distance and published on the competition website.
A. Top 3 submissions for Task 1
The listed Task 1 submissions use image-classification ensembles, feature-enhanced networks, and spatially normalized recognition architectures with recurrent and attention components.
- BASELINE v1 uses image-classification methods and their ensemble.
- Amap_CVLab adds residual and squeeze-and-excitation blocks to avoid lower-dimensional feature collapse.
- Amap_CVLab trains on ReCTS-25k together with additional data.
- TPS-ResNet v1 uses a thin-plate-spline spatial transformer to normalize inputs, followed by ResNet, BiLSTM, and attention.
C. Top 3 submissions for Task 3
The listed Task 3 detectors combine discretization, multi-scale evaluation, model ensembles, two-stage architectures, feature pyramids, and pretrained backbones.
- SANHL_v4 uses sequential-free box discretization to localize text instances, then applies multi-scale testing and model ensembling.
- SANHL_v4 trains with LSVT, ArT, MLT, and ReCTS-25k datasets.
- Tencent-DPPR uses a two-stage detector with multi-scale training and ResNet101 as its backbone.
- Tencent-DPPR uses feature pyramid layers and an LSVT-pretrained model rather than selecting one feature layer by box size.
- Amap-CVLab bases its detector on Mask R-CNN and trains with RCTW, ICDAR2017-MLT, LSVT, and ReCTS-25k.
D. Top 3 submissions for Task 4
Task 4 combines text-line detection with orientation-dependent recognition, using shared baseline components from the other tasks. The reported submissions use detector–recognizer pipelines, including ensemble-based recognition and several detection architectures.
- Top submissions: Tencent-DPPR uses a two-stage ResNet101 detector with feature-pyramid layer selection, multi-scale ensemble testing, box voting, and an ensemble recognition model.
- Top submissions: SANHL detects possible text lines and predicts strings with an ensembled recognition model.
- Top submissions: HUST_VLRGROUP combines Mask R-CNN text detection with a CRNN-based string recognition approach.
- Task 4 baseline: Task 4 first detects text lines, then recognizes horizontal lines as text lines and vertical lines through character detection and recognition.
- Task 4 baseline: The baseline reuses Task 3 for text-line detection, Task 1 for character recognition, and Task 2 for text-line recognition.
VI. CONCLUSIONS
The competition released a large-scale Chinese signboard dataset, established four recognition and detection tasks, and introduced multi-GT evaluation for Chinese text ambiguity. Broad participation produced hundreds of submissions from 46 teams.
- VI. CONCLUSIONS: The ReCTS competition released 25,000 challenging natural scene signboard images and established four tasks.The tasks cover character recognition, text line recognition, text line detection, and end-to-end recognition.
- VI. CONCLUSIONS: Multi-GT evaluation was proposed to address ambiguity in Chinese text evaluation.
- VI. CONCLUSIONS: Hundreds of submissions from 46 teams demonstrated broad community interest in the challenge.
- VI. CONCLUSIONS: The organizers planned to make evaluation scripts available online so users could receive results shortly after submission.