Source-linked AI summary
Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognition
Chee Kheng Chng, Chee Seng Chan
TL;DR
Total-Text addresses the near absence of curved text in established scene text datasets by introducing a dataset that includes curved-oriented text. It also evaluates fine-tuned DeconvNet on this setting and reports a diverse benchmark with 9,330 annotated texts and more than half of images containing at least two orientations.
Problem
Curved text is common in real-world scenes but has close to zero representation in established scene text datasets, limiting evaluation of curved-oriented text detection.
Method
The paper constructs Total-Text with tightly fitted polygon annotations and fine-tunes DeconvNet to analyze segmentation-based text detection on curved text.
Results
9,330 annotated texts and 1.8 orientations per image characterize Total-Text, with more than half of its images containing two or more orientations.
Takeaways & Limitations
Total-Text establishes a benchmark for curved and multi-oriented scene text, while the analysis identifies spatial resolution and contextual information as crucial for segmentation-based methods.
Abstract
from arXiv · showhide
Text in curve orientation, despite being one of the common text orientations in real world environment, has close to zero existence in well received scene text datasets such as ICDAR2013 and MSRA-TD500. The main motivation of Total-Text is to fill this gap and facilitate a new research direction for the scene text community. On top of the conventional horizontal and multi-oriented texts, it features curved-oriented text. Total-Text is highly diversified in orientations, more than half of its images have a combination of more than two orientations. Recently, a new breed of solutions that casted text detection as a segmentation problem has demonstrated their effectiveness against multi-oriented text. In order to evaluate its robustness against curved text, we fine-tuned DeconvNet and benchmark it on Total-Text. Total-Text with its annotation is available at https://github.com/cs-chan/Total-Text-Dataset
(EXTENDED VERSION)
Total-Text addresses the lack of curved text in established scene text datasets by providing a dataset spanning horizontal, multi-oriented, and curved text. It also examines segmentation-based detection and reports difficulty detecting curved text with existing solutions.
- Curved text is common in real-world scenes but largely absent from established datasets such as ICDAR and MSRA-TD500.CUTE80 was identified as the only available curved-text dataset, but it contains only 80 images and minimal scene diversity.
- 1,555 scene images and 9,330 annotated words cover horizontal, multi-oriented, and curved text.
- Total-Text was created to fill the dataset gap in text orientations and support research on curved-text detection.
- Segmentation-based methods were examined because they had reported success on multi-oriented and some curved text without relying on orientation heuristics.
- Current state-of-the-art solutions could not detect curved text effectively in the illustrated examples.
II. RELATED WORKS
The related-work section surveys scene text datasets and detection systems, emphasizing the development of datasets and methods for varied text orientations.
- The section reviews scene text datasets and text detection systems as the paper’s related work.
- It recommends an additional reference for readers seeking a more complete treatment of the field.
A. Scene Text Datasets
Scene text datasets progressed from predominantly horizontal text toward arbitrary and curved orientations, but they differ substantially in scale, diversity, and annotation format.
- ICDAR’03, ICDAR’11, and ICDAR’13 contain horizontal English scene text, with ICDAR’13 comprising 462 images.
- ICDAR’15 introduced 1,670 wearable-device images with arbitrary orientations and frequent out-of-focus text.
- MSRA-TD500 contains 300 training and 200 testing images annotated with minimum-area rectangles for arbitrarily oriented text.
- COCO-Text contains 63,686 images and 173,589 labeled text regions spanning horizontal, arbitrary, and curved orientations, but uses axis-oriented rectangular groundtruth.
- CUTE80 provides curved text but has only 80 images and limited scenery.
- Detection research evolved from component-based methods such as SWT and MSER toward CNN and FCN approaches using contextual information and text-region segmentation.
III. TOTAL-TEXT DATASET
Total-Text addresses the scarcity of curved text data with a manually annotated dataset spanning horizontal, multi-oriented, and curved text. Its polygon groundtruth and diverse orientation compositions are designed to support evaluation beyond conventional orientation assumptions.
- Dataset motivation and scope: 4,265 of 9,330 text instances are curved, addressing their close-to-zero representation in several existing datasets.
- Dataset diversity: The dataset includes varied orientation compositions and scenery, including horizontal, multi-oriented, horizontal-curve, vertical-curve, circular, and wavy text.
- Evaluation target: Orientation-based grouping and straight-line mechanisms can fail on curved words because characters may have multiple orientation variations.
- Annotation design: Polygon-shaped groundtruth tightly fits text regions, whereas rectangular boxes include substantial background for curved text.
B. Dataset Statistics
Total-Text is a diverse scene text dataset designed around curved text while retaining horizontal and multi-oriented instances. Its 1,555 images and 9,330 annotated words combine substantial orientation and scene diversity.
- Total-Text contains 1,555 images split into 1,255 training and 300 testing images.
- 9,330 annotated texts yield an average of 6 instances per image, while more than half of images contain at least two orientations.The dataset averages 1.8 orientations per image.
- Approximately half of the text instances are curved, with the remainder divided nearly equally between horizontal and multi-oriented text.Curved instances include horizontal curved, vertical curved, circular, and wavy types.
- Curved text commonly co-occurs with horizontal or multi-oriented text, creating mixed-orientation images that challenge robustness and generalization.
- Business locations, branding logos, and merchant stores contain 61.2% of curved text instances, while tourist spots account for 21.1%.
IV. SEMANTIC SEGMENTATION FOR TEXT DETECTION
The paper investigates semantic-segmentation-based text detection as a way to assess performance on curved text. It fine-tunes DeconvNet and evaluates it on Total-Text.
- The authors fine-tune DeconvNet and evaluate it on Total-Text to investigate segmentation-based detection for curved text.
- Prior segmentation-based methods achieved state-of-the-art results and showed successful curved-text detection in limited examples.
- The study is motivated by a segmentation approach that achieved good results without heuristic grouping rules.
A. DeconvNet
DeconvNet is used as an investigation tool because its segmentation performance and multiple deconvolutional layers support detailed analysis. The authors convert and fine-tune it for text localization rather than propose a new curved-text solution.
- DeconvNet was selected for its state-of-the-art semantic-segmentation results on Pascal VOC and its multiple deconvolutional layers.
- The paper converts DeconvNet for text localization and explicitly does not propose a new solution to the curved-text problem.
- The original 21-class output is reduced to two layers representing text and non-text, followed by one-step fine-tuning.
- The network is pre-trained on legible COCO-Text data using binary masks, with approximately 15k cropped training examples.
B. Experiments
Inference resizes images, thresholds the saliency map, and groups text pixels into polygons. Experiments show that DeconvNet finds text across orientations, while repeated-pattern backgrounds, word grouping, and curved-text confidence remain problematic.
- B. Experiments: Input images are resized to 224×224, saliency maps are thresholded at 0.5, and connected components are tightly bounded with polygons.
- B. Experiments: Failure cases arise from repeated-pattern backgrounds and grouping multiple word candidates into one output.
- B. Experiments: The authors attribute limited robustness partly to loosely bounded training labels and identify missing word-level segmentation as a problem.
- B. Experiments: DeconvNet finds text candidates across all orientations without orientation assumptions or heuristic grouping.
- B. Experiments: Lower network layers highlight regions, while deeper layers develop finer details that can reveal characters in curved text.
- B. Experiments: Text-line supervision is proposed as a possible improvement for uncertain curved-text shapes; one cited method's F-score fell from 0.84 to 0.5 without FTN.
V. CONCLUSION
Total-Text introduces curved text into scene text datasets and evaluates segmentation-based detection on this previously underrepresented orientation. The authors identify feature-map resolution and contextual information as crucial for handling curved and multi-oriented text.
- Total-Text addresses the lack of curved text in existing scene text datasets.
- The dataset treats curved text as part of the multi-oriented text detection problem.
- The authors fine-tuned and analyzed DeconvNet’s response to curved text.
- Spatial resolution of feature maps and contextual information appeared crucial for segmentation-based methods.
- Segmentation-based methods can predict text regions across orientations without hard-coded rules.
VI. APPENDIX
The appendix presents challenging scene conditions, varied text orientations, and evidence that polygon-shaped groundtruth changes evaluation outcomes. Polygon annotations provide tighter regions and a more accurate representation of detection performance.
- Challenging Examples: Total-Text includes perspective distortion, font and size variation, text-like backgrounds, uneven lighting, and low contrast.
- Text Orientations: The dataset spans unified, dual, and highly varied text orientations across images.
- Different Groundtruth Evaluations: Polygon-shaped groundtruth provides a more accurate evaluation representation than rectangle-shaped groundtruth.
- Different Groundtruth Evaluations: Rectangle-shaped groundtruth can yield higher precision when detected regions overlap only part of the actual text region.
- Different Groundtruth Evaluations: Missed large intersection areas with polygon groundtruth can lower both recall and precision.
C. Groundtruth Examples
Total-Text annotations encode spatial location, transcript, orientation, and binary text masks. Polygon-shaped groundtruth tightly bounds text regions and represents annotations more comprehensively than rectangular regions.
- Each image is annotated with spatial locations, transcripts, text-instance orientations, and binary masks.
- Polygon-shaped groundtruth can disagree with rectangle-shaped regions when representing text instances.
- Polygon-shaped bounding boxes tightly bound the text and make annotations more comprehensive.