PaperlayerView exact revision on arXiv

Source-linked AI summary

ContourNet: Taking a Further Step toward Accurate Arbitrary-shaped Scene Text Detection

Yuxin Wang, Hongtao Xie, Zhengjun Zha, Mengting Xing, Zilong Fu, Yongdong Zhang

arXiv:2004.04940v1cs.CV

TL;DR

Arbitrary-shaped scene text detection is challenged by false positives and large scale variance. ContourNet addresses these issues with scale-insensitive proposal generation and orthogonal local texture modeling, achieving strong benchmark performance without external training data.

  • Problem

    False positives in text representations and large scale variance make accurate arbitrary-shaped scene text detection difficult.

  • Method

    ContourNet combines IoU-driven Adaptive-RPN proposals with LOTM, which models local texture in two orthogonal directions and represents text with contour points.

  • Results

    85.4% and 83.9% F-measure are reported on Total-Text and CTW1500, respectively, at 3.8 FPS and 4.5 FPS without external training data.

  • Takeaways & Limitations

    Considering orthogonal texture responses and scale-invariant proposal training yields more accurate arbitrary-shaped text descriptions across challenging benchmarks.

Abstract

from arXiv · show

Scene text detection has witnessed rapid development in recent years. However, there still exists two main challenges: 1) many methods suffer from false positives in their text representations; 2) the large scale variance of scene texts makes it hard for network to learn samples. In this paper, we propose the ContourNet, which effectively handles these two problems taking a further step toward accurate arbitrary-shaped text detection. At first, a scale-insensitive Adaptive Region Proposal Network (Adaptive-RPN) is proposed to generate text proposals by only focusing on the Intersection over Union (IoU) values between predicted and ground-truth bounding boxes. Then a novel Local Orthogonal Texture-aware Module (LOTM) models the local texture information of proposal features in two orthogonal directions and represents text region with a set of contour points. Considering that the strong unidirectional or weakly orthogonal activation is usually caused by the monotonous texture characteristic of false-positive patterns (e.g. streaks.), our method effectively suppresses these false positives by only outputting predictions with high response value in both orthogonal directions. This gives more accurate description of text regions. Extensive experiments on three challenging datasets (Total-Text, CTW1500 and ICDAR2015) verify that our method achieves the state-of-the-art performance. Code is available at https://github.com/wangyuxin87/ContourNet.

1. Introduction

ContourNet targets false positives and large scale variance in arbitrary-shaped scene text detection. It combines scale-insensitive proposals with orthogonal local texture modeling to improve contour localization and detection accuracy.

  • Motivation: Scene text detection must handle complex backgrounds, arbitrary shapes, and large variation in color, texture, and scale.Accurate detection supports end-to-end text recognition and real-world applications.
  • Motivation: False positives arise because arbitrary-orientation texture modeling can respond strongly to non-text patterns with text-like characteristics.CNN-based convolutional modeling may produce high responses for patterns such as streaks.
  • Motivation: Orthogonal texture responses suppress false positives because strongly unidirectional patterns are weakly activated in the orthogonal direction.ContourNet therefore models local texture horizontally and vertically, using both responses jointly.
  • Motivation: Large scale variance makes scene-text samples difficult for CNN-based methods to learn.ContourNet addresses this through a scale-invariant metric rather than aggregating multi-scale features.
  • Proposed approach: Adaptive-RPN generates proposals using boundary points and IoU-driven training, making proposal generation insensitive to text scale and more shape-aware.The predicted points indicate the spatial extent of each text instance.
  • Proposed approach: LOTM models proposal texture in two orthogonal directions and represents text regions with contour points, followed by point re-scoring.The method is designed to produce high-quality contour points while suppressing false positives.
  • Results: 85.4% and 83.9% F-measure are achieved on Total-Text and CTW1500, respectively, at 3.8 FPS and 4.5 FPS without external training data.The authors report that these results outperform recent counterparts by a large margin.

2. Related Works

Related methods use regression or segmentation representations for text localization. ContourNet combines a two-stage architecture with contour-point representation, aiming to retain their advantages while improving scale handling and false-positive suppression.

  • Existing approaches: Scene text detection methods are broadly divided into regression-based and segmentation-based approaches.Regression methods predict offsets to localize text boxes, while segmentation methods describe regions and reconstruct instances through postprocessing.
  • Regression-based methods: Regression methods localize text using offsets from anchors or pixels, with some methods reconstructing arbitrary-shaped instances from predicted points.Examples include corner-point prediction and recurrent refinement of point pairs.
  • Segmentation-based methods: Segmentation methods represent text regions through pixel links, segmentation maps, or clustering procedures before reconstructing text instances.These representations are designed to separate adjacent or dense text instances.
  • ContourNet: ContourNet uses a two-stage architecture and contour points to integrate advantages of regression and segmentation methods.Adaptive-RPN and false-positive suppression are reported to improve scale handling and text-region description.

3. Proposed Method

ContourNet combines Adaptive-RPN, LOTM, and point re-scoring in a two-stage detector to address scale variance, false positives, and arbitrary-shaped text localization.

  • Adaptive-RPN: Adaptive-RPN generates finer text-region localization by bounding the spatial extent of several refined points rather than modeling only a rectangular scope.The point representation automatically accounts for shape and semantically important local areas without additional supervision.
  • Adaptive-RPN: Adaptive-RPN replaces conventional 4-d regression with refined predefined points and IoU-based optimization for scale-insensitive proposal generation.Refined points are bounded into predicted boxes, while the center point normalizes the box representation.
  • LOTM: LOTM uses parallel 1×k and k×1 convolutions to model horizontal and vertical local textures, producing two normalized heatmaps of contour points.The receptive-field size k controls the modeled texture range, while the two branches preserve efficiency through localized operations.
  • Point Re-scoring Algorithm: Point Re-scoring applies NMS to both directional heatmaps and retains contour candidates with high responses in both orthogonal directions.This suppresses predictions characterized by strong unidirectional or weakly orthogonal texture responses.
  • Training Objective: The training objective combines Adaptive-RPN classification and regression, contour-point losses in two directions, and box classification and regression losses.The loss terms are balanced with weights, while the Adaptive-RPN regression uses IoU loss and LOTM uses class-balanced cross-entropy.

4.1. Datasets

The evaluation uses Total-Text, CTW1500, and ICDAR2015, covering horizontal, multi-oriented, curved, and arbitrarily oriented scene text with different annotation structures.

  • ICDAR2015 contains 1000 training and 500 testing images with word-level annotations represented by four vertices for arbitrarily oriented texts.
  • CTW1500 contains 1000 training and 500 testing images for curve text detection, annotated with 14 boundary points at text-line level.
  • Total-Text contains 1255 training and 300 testing images with word-level annotations spanning horizontal, multi-oriented, and curved texts.

4.2. Implementation Details

The model is trained separately on each dataset using ImageNet-pretrained ResNet50, standard geometric augmentation, fixed training schedules, and dataset-specific test resolutions.

  • Training uses an ImageNet-pretrained ResNet50 backbone, PyTorch, Adam, one NVIDIA TITANX GPU, and only each dataset’s official training images.
  • Data augmentation applies random rotation, random horizontal flipping, and random cropping during training.
  • Models train for 180k iterations, starting at 2.5 × 1e −3 and reducing the learning rate after 120k and 160k iterations.
  • Polygon annotations are converted into two-points-wide edges with SciPy’s distance transform, and edge points supervise contour learning in both directions.
  • Testing uses single-scale inputs with images resized to 720×1280 for Total-Text and CTW1500 and 1200 × 2000 for ICDAR2015.Alpha-Shape generates bounding boxes from contour point candidates under official evaluation protocols.

4.3. Ablation Study

Ablations show that Adaptive-RPN improves detection across text scales, while LOTM benefits from local texture modeling with compact orthogonal kernels.

  • Adaptive-RPN: Using 9 pre-defined points in Adaptive-RPN improves F-measure by 0.6%.The remaining experiments use n = 9.
  • Adaptive-RPN: Adaptive-RPN improves F-measure by 0.9% on Total-Text and 0.8% on CTW1500 over conventional RPN.On Total-Text, improvements are 1.4%, 0.3% and 1.1% for small-, middle- and large-size texts, respectively.
  • Adaptive-RPN: Adaptive-RPN improves Total-Text F-measure by 1.4%, 0.3% and 1.1% for small-, middle- and large-size texts, respectively.The comparison considers valid pairs within the same size category.
  • LOTM: LOTM reaches 85.4% F-measure with 1×3 and 3×1 convolutional kernels, while larger receptive fields reduce performance.The authors attribute the decline to additional noise and use kernel size 3 thereafter.
  • LOTM: Orthogonal texture modeling improves recall, precision and F-measure by 1.2%, 1.6% and 1.4%, respectively, over jointly modeled arbitrary orientations.Single-direction modeling achieves 80.6% versus 85.4% F-measure for LOTM.

4.4. Comparisons with State-of-the-Art Methods

ContourNet is evaluated on curved, long curved and multi-oriented text benchmarks, achieving strong single-scale results without external training data. Its reported F-measures are 85.4% on Total-Text, 83.9% on CTW1500 and 86.9% on ICDAR2015.

  • 4.4.1 Evaluation on Curved Text Benchmark: On Total-Text, ContourNet achieves 83.9% recall, 86.9% precision and 85.4% F-measure without external data.The paper reports that it outperforms existing state-of-the-art methods including LOMO, PAN and PSE.
  • 4.4.1 Evaluation on Curved Text Benchmark: ContourNet runs at 3.8 FPS on Total-Text and exceeds CRAFT and LOMO by 1.8% and 2.1% F-measure, respectively.The comparison uses only official training data and single-scale testing for ContourNet.
  • 4.4.2 Evaluation on Long Curved Text Benchmark: On CTW1500, ContourNet achieves 83.9% versus 80.8% F-measure for LOMO.It also reports relative improvements of 5.8% in recall and 2.4% in F-measure over MSR without external training data, while running at 4.5 FPS.
  • 4.4.3 Evaluation on Multi-oriented Text Benchmark: On ICDAR2015, ContourNet achieves 86.9% F-measure, compared with 87.6% for Wang et al. and 86.8% for that method without SE blocks.The evaluation uses single-scale results.
  • Qualitative Results: Figure 5 presents detection results for Total-Text, CTW1500 and ICDAR2015 in panels (a), (b) and (c), respectively.The panels correspond to curved, long curved and multi-oriented text evaluations described in the surrounding results.

4.5. Effectiveness of ContourNet

ContourNet improves arbitrary-shaped text detection through finer, scale-aware proposal localization and orthogonal texture-based false-positive suppression. Its benefits are reflected in both qualitative analyses and benchmark F-measures.

  • Effectiveness of Adaptive-RPN: Adaptive-RPN achieves finer text-region localization than conventional RPN when regression distances are large or target and default boxes have different ratios.Its shape awareness and scale-invariant training objective improve localization in these cases.
  • Effectiveness of false-positive suppression: The false-positive suppression ratio remains considerable as θ varies from 0.1 to 0.9, indicating stronger suppression than false-negative introduction.The method is reported as more effective at suppressing false positives than causing false negatives.
  • Effectiveness of false-positive suppression: Retained positive points with strong texture responses in both orthogonal directions accurately represent text regions despite causing few false negatives.The orthogonal response criterion is illustrated qualitatively in Fig.1.

5. Conclusion

ContourNet addresses false positives in text representations and large scale variance through Adaptive-RPN, LOTM, and Point Re-scoring Algorithm. Experiments on public benchmarks cover long, curved, and oriented text cases.

  • Conclusion: ContourNet combines Adaptive-RPN, LOTM, and Point Re-scoring Algorithm to localize, represent, and filter arbitrary-shaped text regions.Adaptive-RPN uses semantic points, LOTM models orthogonal local texture with contour points, and point re-scoring considers both orthogonal responses.
  • Conclusion: The method is evaluated on public benchmarks containing long, curved, and oriented text cases.
  • Conclusion: Future work aims to develop an end-to-end text reading system.
Loading 2004.04940v1…