Meta-learning Convolutional Neural Architectures for Multi-target Concrete Defect Classification with the COncrete DEfect BRidge IMage Dataset

Martin Mundt; Sagnik Majumder; Sreenivas Murali; Panagiotis Panetsos; Visvanathan Ramesh

Source-linked AI summary

Meta-learning Convolutional Neural Architectures for Multi-target Concrete Defect Classification with the COncrete DEfect BRidge IMage Dataset

Martin Mundt, Sagnik Majumder, Sreenivas Murali, Panagiotis Panetsos, Visvanathan Ramesh

arXiv:1904.08486v1Apr 2, 2019cs.CVcs.LGstat.ML

TL;DR

Concrete bridge defect recognition must handle small, overlapping defects and substantial variation in material appearance, illumination, weather, and surface markings. The paper introduces CODEBRIM and adapts MetaQNN and ENAS to discover CNN architectures for this multi-target task. The resulting architectures generally outperform most baseline CNNs while using fewer parameters, reaching up to 72% test accuracy with fewer than 5 million parameters.

Problem
Concrete bridge defect recognition lacks sufficiently diverse multi-target evidence despite small, overlapping defects and strong environmental and appearance variation.
Method
The paper introduces CODEBRIM and compares literature CNNs with MetaQNN and ENAS architecture search adapted for multi-target defect classification.
Results
Meta-learned architectures outperform most baseline CNNs and achieve up to 72% test accuracy with fewer than 5 million parameters.
Takeaways & Limitations
Task-specific meta-learned architectures provide more parameter-efficient and accurate models than most human-designed CNN baselines on CODEBRIM.
Takeaways & Limitations
ImageNet and MINC transfer-learning experiments did not improve results, and bridge-based dataset splitting was infeasible to balance defect occurrences uniformly.

Abstract

from arXiv · show

Recognition of defects in concrete infrastructure, especially in bridges, is a costly and time consuming crucial first step in the assessment of the structural integrity. Large variation in appearance of the concrete material, changing illumination and weather conditions, a variety of possible surface markings as well as the possibility for different types of defects to overlap, make it a challenging real-world task. In this work we introduce the novel COncrete DEfect BRidge IMage dataset (CODEBRIM) for multi-target classification of five commonly appearing concrete defects. We investigate and compare two reinforcement learning based meta-learning approaches, MetaQNN and efficient neural architecture search, to find suitable convolutional neural network architectures for this challenging multi-class multi-target task. We show that learned architectures have fewer overall parameters in addition to yielding better multi-target accuracy in comparison to popular neural architectures from the literature evaluated in the context of our application.

1. Introduction

Concrete bridge defect recognition is difficult because defects vary widely, overlap, and appear under changing environmental conditions. The paper addresses this challenge with CODEBRIM and task-specific meta-learned CNN architectures.

Motivation: Concrete bridge defects can be small, overlapping, and difficult to distinguish amid variation in material appearance, lighting, weather, and surface alterations.These factors make multi-class multi-target recognition relevant to structural-safety assessment.
Motivation: Deep learning has excelled on related recognition tasks, but concrete bridge defect recognition has largely remained focused on cracks and traditional approaches.The paper identifies limited use of deep learning beyond crack-oriented recognition.
Contributions: CODEBRIM provides a high-resolution, multi-target dataset with six mutually non-exclusive classes, including five defects and non-defective background.The classes are crack, spallation, efflorescence, exposed bars, corrosion stains, and non-defective background.
Contributions: The study evaluates established CNN architectures and adapts MetaQNN and ENAS to search for architectures tailored to multi-target concrete defect classification.The comparison covers both literature baselines and reinforcement-learning-based architecture search methods.
Contributions: Meta-learned architectures achieve higher accuracy and lower parameter counts than the presented human-designed CNN counterparts.The paper frames this result as improved task performance with greater parameter efficiency.

2. Prior and related work

Prior computer-vision benchmarks and concrete-defect studies largely emphasize single-target recognition, crack detection, and established CNN baselines. The paper situates CODEBRIM as a multi-target, context-rich alternative and uses CNN architectures as comparison points.

Datasets: Major image and scene benchmarks such as ImageNet, Pascal VOC, SUN, and MINC predominantly frame recognition as assigning a specific class to an image, area, or pixel.These datasets underpin much recent computer-vision deep-learning research.
Defect recognition: Concrete defect studies have mostly used task-specific models for crack-versus-non-crack classification, often with limited clutter and structural context.Some work adds spalling, but the reviewed studies still focus on single-target scenarios.
Convolutional neural networks: The paper compares literature CNN families including AlexNet, VGG, texture-CNN, FV-CNN, WRN, and DenseNet as references for task-specific architectures.These models represent successive architectural strategies such as depth, feature pooling, residual connections, and dense connectivity.
Meta-learning neural architectures: Neural architecture meta-learning treats architecture design as black-box optimization over weight training to find task-specific network designs.This approach is presented as an alternative to relying solely on human design intuition.

3. The CODEBRIM dataset

CODEBRIM was collected to represent overlapping concrete defects in realistic bridge contexts, including variation in scale, resolution, weather, and accessibility. Its high-resolution images and bounding-box annotations support multi-target defect classification.

Dataset motivation and composition: CODEBRIM contains five common defect categories collected from 30 unique bridges under varied scales, resolutions, weather conditions, cameras, and viewing contexts.The categories are crack, spallation, exposed reinforcement bar, efflorescence, and corrosion stains.
Image acquisition and camera choice: Cracks as narrow as 0.3 mm motivated high-resolution acquisition and camera choices constrained by pixel-level defect resolution.The paper estimates roughly 0.1 mm per pixel under an example consumer-camera setup.
Annotation: The dataset uses a multi-stage annotation process with defect bounding boxes labeled sequentially by class.This process followed image curation and produced the dataset properties reported by the authors.
Dataset properties: 1590 high-resolution images from 30 bridges, 5354 annotated defect boxes, and 2506 generated non-overlapping background boxes comprise the dataset.The defect boxes are largely overlapping, matching the dataset’s multi-target design.
Dataset properties: The dataset includes 2507 cracks, 1898 spallation instances, 833 efflorescence instances, 1507 exposed-bar instances, and 1559 corrosion-stain instances.These counts describe the five defect classes used in the dataset.
Multi-target variability: Multiple defect classes commonly occur in one bounding-box annotation, while defect boxes vary substantially in aspect ratio, scale, and resolution.Cracks can be fine and elongated, whereas spalled areas can vary almost arbitrarily.

4. Meta-learning convolutional neural networks for multi-target defect classification

The study adapts MetaQNN and ENAS to discover CNN architectures for CODEBRIM’s challenging multi-target concrete-defect classification task. Meta-learned models achieve strong accuracy with fewer parameters than most literature CNN baselines, although the two search strategies show no significant performance difference.

Task and evaluation: The study uses CODEBRIM to investigate CNN architectures tailored to multi-target defect classification, while excluding bounding-box detection from evaluation.The dataset is split into training, validation, and final test sets for comparing meta-learned and literature CNN architectures.
Task and evaluation: Multi-target accuracy counts a prediction as correct only when every target is classified correctly, using per-class sigmoid outputs thresholded at 0.5.Training uses binary cross entropy, while class balancing and fixed-size crops address imbalance and scale variation.
Meta-learning method: The adapted MetaQNN search adds stride-based downsampling, spatial-pyramid pooling, selectable hidden-layer sizes, batch normalization, ReLU activations, and ResNet-like skip connections.These extensions make MetaQNN more comparable to ENAS, which already includes several of these design choices.
Meta-learning method: Both MetaQNN and ENAS learn to propose architectures with improving multi-target validation accuracy after exploration, while ENAS requires final end-to-end retraining because it uses shared weights.MetaQNN individual architectures are evaluated directly during the search, whereas ENAS’s final architecture accuracies require retraining.
Results and discussion: The results suggest that task-specific meta-learned architectures can be better suited to multi-target concrete-defect classification than architectures designed for other datasets.The authors note that improvements among ImageNet CNNs do not transfer similarly to this task and that the two search strategies do not differ significantly in performance.

5. Conclusion

The paper introduces CODEBRIM and task-specific meta-learned architectures for multi-target concrete defect classification. These architectures use fewer parameters and layers while achieving higher accuracy than human-designed counterparts, with best test accuracies up to 72%.

CODEBRIM is a novel multi-class multi-target dataset for concrete defect recognition.
The study compares MetaQNN and ENAS to identify task-specific neural architectures.The approaches are reinforcement-learning-based neural architecture search methods.
Meta-learned architectures have fewer parameters, fewer layers, and higher accuracy than human-designed counterparts on the multi-target classification task.
72% is the highest reported multi-target test accuracy achieved by the meta-learned models.
The supplementary material documents dataset details, meta-learning architectures, transfer-learning experiments, and qualitative classification examples.Transfer-learning experiments with ImageNet- and MINC-pretrained models showed no improvements and were moved to the supplement.

B. CODEBRIM dataset

The CODEBRIM dataset combines high-resolution bridge images with multi-stage expert-assisted annotations for overlapping concrete defects and background regions. Its acquisition uses varied cameras and settings, while its annotation process labels defect classes separately within bounding boxes.

Delamination occurrences are labeled with cracks because single-camera views often cannot visually distinguish their boundaries from cracks.
CODEBRIM includes high-resolution images acquired with four cameras using varied resolutions, focal lengths, apertures, illumination, and imaging conditions.The cameras include resolutions above Full-HD, with the highest reaching 6000 × 4000 pixels.
The dataset’s aspect ratio remains almost constant across absolute resolutions, and most images were acquired at very high resolutions.
A multi-stage annotation process first marks defect bounding boxes, then labels each defect class separately within every box.The staged procedure was intended to reduce annotation errors by focusing annotators on one defect class at a time.
The camera description records resolution, exposure time, focal length, aperture, ISO speed, and flash information.
Background bounding boxes are sampled according to the counts, aspect ratios, and sizes of annotated defect bounding boxes.

B.4. Further dataset statistics

Supplementary statistics characterize CODEBRIM’s multiple defect locations, varied bounding-box geometry, and complex background sampling. These distributions reflect the dataset’s overlapping defects, diverse scales, aspect ratios, and scene conditions.

A bounding box commonly contains more than one defect class, and images generally contain multiple defect locations.
The supplementary statistics support future semantic-segmentation extensions by documenting multiple defect locations within acquired images.
Per-class bounding-box size and aspect-ratio distributions are reported separately, but they are not mutually exclusive because annotations can contain multiple defects.
The dataset’s variation spans defect geometry, bounding-box scale and aspect ratio, camera pose, illumination, and target structure.Cracks can be fine and elongated, whereas spalled areas can vary widely in shape.
Background samples follow the aspect-ratio trend of defect boxes while remaining fewer than the integrated total number of defect boxes.Background boxes were also checked to avoid overlap with defect boxes and to remain evenly distributed among images.

C.1. Per-class and multi-target CNN accuracies

The paper argues that multi-target accuracy is the appropriate evaluation target because overlapping defects affect structural degradation severity. Per-class averages can favor models that neglect these overlaps.

Single-class accuracies and their average do not adequately represent the multi-target concrete-defect task.
Multi-target accuracy better reflects cases where multiple defect classes occur and overlap because such overlap magnifies degradation severity.
Rewards based on average per-class accuracy can produce models that predict only a subset of classes correctly and neglect overlaps.

C.2. Meta learned architecture definitions

The appendix details the top three MetaQNN and ENAS architectures found for multi-target concrete defect classification, including their layer operations and skip-connection structure.

The top three MetaQNN and ENAS architectures are documented because their accuracies are reported in the main body.
MetaQNN architectures specify convolutional filter sizes, filter counts, padding, strides, skip connections, and spatial pyramidal pooling scales.
ENAS architectures use seven convolutional layers followed by a linear transformation and include numerous skip connections represented as acyclic graphs.

C.3. Transferring ImageNet and MINC features

The paper evaluates transfer learning from ImageNet and MINC features for multi-target defect classification, while noting that generic transferred features may not suit the combined object, texture, and multi-target demands.

Transfer learning fine-tunes CNNs with ImageNet- and MINC-pretrained convolutional features while training only the classification stage for 70 epochs.The reported validation and test results are summarized in Table 6.
Table 4 defines per-class, average, validation, and test accuracy measures used for evaluating the defect classifiers.
Table 5 identifies the top three MetaQNN architectures used for comparison in the task.
The authors hypothesize that transfer learning may require selecting relevant pretrained features or using different fine-tuning strategies for this multi-target task.The concern arises because the task combines object and texture recognition demands.

C.4. Classification examples

Qualitative validation examples illustrate correct and erroneous multi-target predictions, showing that the task includes varied defect combinations and difficult visual conditions.

The qualitative analysis separates validation predictions into correct classifications, partially correct classifications, and classifications recognizing none of the present defects.
Correct examples contain all ground-truth labels, whereas partially correct examples miss or add one or more defect classes.
The examples are paired with ground-truth labels and model predictions in Figure 12 to show varied defect types and combinations.
Most multi-target predictions are correct, but partially visible defects and other visual factors still make classification difficult.

C.5. Alternative dataset splits

The paper examines bridge-based dataset splits as an alternative evaluation setting, while explaining why balanced splits are difficult and reporting that meta-learned architectures continue to outperform baselines.

Split constraints: A bridge-based split cannot generally provide a balanced class distribution because bridges differ in defect counts, classes, severity, and overlap.
Split constraints: Excluding frequent defects or accepting approximate balance can omit difficult or easy instances, bias class preference, or distort accuracy interpretation.
Alternative bridge-based split: Bridge-based splitting can expose overfitting to bridge-specific properties such as surface roughness, color, context, and changing scene conditions.
Alternative bridge-based split: The alternative split uses three bridges for validation and three for testing, and retrains the architectures without repeating hyperparameter search or architecture sampling.
Results: The alternate-split accuracies are similar to the main experiments, and meta-learned architectures still outperform baselines despite changes in model ordering.