PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep learning in automated ultrasonic NDE -- developments, axioms and opportunities

Sergio Cantero-Chinchilla, Paul D. Wilcox, Anthony J. Croxford

arXiv:2112.06650v2eess.SP

TL;DR

Ultrasonic NDE has traditionally depended on operator interpretation and rudimentary automation, while DL research lacks consensus on requirements for dependable autonomous use. This paper reviews DL applications by NDE task, proposes automation levels and axioms, and identifies current capabilities and remaining challenges. It concludes that DL supports processing, operator-independent characterisation, and repetitive detection, but practical applicability is constrained by heterogeneous development and unresolved reliability and uncertainty issues.

  • Problem

    Ultrasonic NDE remains largely operator-driven, and the field lacks consensus on the requirements and direction of DL methods for autonomous inspection.

  • Method

    The paper reviews DL-based NDE research by task, proposes an automation roadmap, and derives axioms from literature, regulations, and industrial input.

  • Results

    DL has been used for processing tasks, reducing dependence on operator experience, and automating repetitive defect detection, but development practices are heterogeneous.

  • Takeaways & Limitations

    The roadmap and axioms provide a framework for setting higher automation objectives and standardizing DL-based inspection development.

  • Takeaways & Limitations

    Full automation requires reliability quantification under diverse uncertainties, while chained models require uncertainty propagation to be quantified.

Abstract

from arXiv · show

The analysis of ultrasonic NDE data has traditionally been addressed by a trained operator manually interpreting data with the support of rudimentary automation tools. Recently, many demonstrations of deep learning (DL) techniques that address individual NDE tasks (data pre-processing, defect detection, defect characterisation, and property measurement) have started to emerge in the research community. These methods have the potential to offer high flexibility, efficiency, and accuracy subject to the availability of sufficient training data. Moreover, they enable the automation of complex processes that span one or more NDE steps (e.g. detection, characterisation, and sizing). There is, however, a lack of consensus on the direction and requirements that these new methods should follow. These elements are critical to help achieve automation of ultrasonic NDE driven by artificial intelligence such that the research community, industry, and regulatory bodies embrace it. This paper reviews the state-of-the-art of autonomous ultrasonic NDE enabled by DL methodologies. The review is organised by the NDE tasks that are addressed by means of DL approaches. Key remaining challenges for each task are noted. Basic axiomatic principles for DL methods in NDE are identified based on the literature review, relevant international regulations, and current industrial needs. By placing DL methods in the context of general NDE automation levels, this paper aims to provide a roadmap for future research and development in the area.

1. Introduction

Ultrasonic NDE has largely relied on operator interpretation supported by rudimentary automation, while DL-based methods motivate broader automation but lack agreed requirements. The paper reviews relevant work, proposes automation levels, and identifies axioms for NDE DL methods.

  • Current state: Traditional ultrasonic NDE relies on trained operators supported by rudimentary tools, except in some simple, precisely specified mass-production inspections.Examples include automatic thickness calculation from A-scan data and amplitude-based flaw detection.
  • Current state: Rudimentary methods work only for well-defined signals and cannot handle complex variations caused by environmental, manufacturing, geometric, or material conditions.
  • Paper scope: The review organizes DL-based NDE research around data pre-processing, detection, characterisation, and property measurement tasks.
  • Paper scope: The paper proposes an automation roadmap with levels that distinguish operator and autonomous-system responsibilities as automation advances.The levels are analogous to automation levels used in domains such as autonomous aircraft.
  • Motivation: The paper identifies axiomatic properties for NDE DL methods because the field lacks consensus on requirements for data-driven approaches.The axioms were informed by literature and discussions with industries spanning aerospace, nuclear energy, renewable energy, and oil and gas.

2. Neural networks and models

The paper introduces common neural-network architectures used to process different data structures and temporal dependencies, then explains how these architectures can be combined for NDE applications.

  • Fully-connected networks: FCNNs transform data through weighted matrix multiplication, bias addition, and an activation function; shallow and deep variants differ in layer count.Shallow FCNNs have three fully connected layers, whereas deep FCNNs contain multiple fully connected layers.
  • Fully-connected networks: Shallow FCNNs support classification and regression but are limited for highly nonlinear data and extracting spatial patterns.
  • Fully-connected networks: Deep FCNNs extract more nonlinear patterns than shallow FCNNs but are inefficient for structured data because they ignore relative input positions.
  • Convolutional networks: CNNs use convolutional layers to extract multidimensional features from structured 1D, 2D, 3D, or 4D data.The convolution computes a dot product between a kernel and the input data.
  • Recurrent networks: RNNs model dynamic information in temporal sequences, while LSTM and GRU variants use memory cells and gates to represent longer dependencies.
  • Autoencoders: Autoencoders use encoder and decoder networks to learn lower-dimensional latent representations and reconstruct input data or a variation of it.They can be trained without supervision and used for denoising images and time series.
  • Hybrid architectures: Base architectures can be combined, such as in convolutional LSTM networks that replace fully connected state updates with convolutions.
  • Architectural overview: Figure 1 distinguishes shallow FCNN, deep FCNN, CNN, RNN, LSTM, and classical AE architectures; green marks inputs, pink outputs, and other colours internal data.

3. Review of the literature

The literature review organizes DL-based NDE contributions by inspection task, covering processing through defect characterisation while recognizing overlap between machine-learning tasks.

  • Review organization: The review covers NDE-focused DL methodologies from data pre-processing, including denoising and imaging, through defect characterisation.
  • Review organization: It identifies common development challenges and illustrates solutions reported in the literature.
  • Review organization: Task-based categorization can overlap because one NDE problem may involve multiple machine-learning tasks.

3.1. Data pre-processing

Data pre-processing in ultrasonic NDE improves or transforms data before later information-extraction tasks, including denoising, feature extraction, compression, and image formation. Reviewed DL approaches address these functions through methods such as autoencoders, CNNs, and recurrent networks.

  • Data pre-processing improves data quality for later NDE information extraction through denoising, feature extraction, compression, and image creation or processing.
  • Denoising of A-scans/images: A CNN trained on 2000 simulated finite-element signals achieved high reflector-detection success and very low false detection rates up to 20dB SNR, after which accuracy degraded sharply.
  • Denoising of A-scans/images: Denoising methods target raw signals, feature extraction, or ultrasonic images, with CNNs and recurrent models used to separate echoes, classify scan signals, and remove image noise.
  • Denoising of A-scans/images: The air-coupled ultrasound denoising method achieved over 90% classification accuracy, with misclassification mainly located at defect edges.
  • Data reduction and compression: DL compression methods reconstruct ultrasonic array signals for imaging, with one GA and FCNN approach running faster and slightly outperforming PCA while autoencoders exploit bottleneck representations.
  • More efficient / higher resolution image formation: An end-to-end model jointly trains 3D CNN filtering, DAS image formation, and 2D CNN segmentation, while image super-resolution can overcome the diffraction limit and support more precise defect characterisation.

3.2. Defect detection

Deep learning has been applied to ultrasonic defect detection across A-scans, B-scans, tomography images, and wavefield images. The reviewed work emphasizes faster or more automated inspection, transfer learning, and comparisons across established architectures.

  • DL detects defects directly from A-scans, which are faster to acquire and generally require cheaper equipment than C-scan images.
  • DL-based detection has been demonstrated on B-scans from train wheels and ultrasonic tomography images from concrete.
  • A benchmark evaluated AlexNet, VGGNet, ResNet-18, Inception-v3, Wide Resnet, DenseNet, and SE-ResNet-50 on 7000 ultrasonic inspection images.The models were compared using detection accuracy, model complexity, memory usage, and computational efficiency.
  • Transfer learning pre-trains DL models on large image databases before fine-tuning them on smaller ultrasonic datasets.
  • The reviewed DL methodologies exceeded classical methods and human operators in reported comparisons, suggesting medium-term automation of repetitive NDE tasks is plausible.

3.3. Defect characterisation

Defect characterisation with DL covers both defect classification and quantitative estimation of defect properties. Existing studies focus mainly on metallic components and welds, while composite structures and regression-based sizing remain less developed.

  • Defect characterisation is organised into classifying defects and quantifying numerical properties such as crack length.
  • FCNNs and CNNs have classified porosity, lack of fusion, lack of penetration, and cracks in steel plates and welded regions.An early example used an FCNN to classify defects from A-scans.
  • A CNN classified voids and delaminations from wavelet-transformed C-scan signals in a CFRP structure.
  • DL defect classification has been used successfully in metallic components, especially welds, but has been scarcely addressed in anisotropic composite plates.Composite structures pose challenges from multiple internal reflections and anisotropy.
  • Physics-based models for relatively large training databases and transfer learning are identified as important for DL in composite structures.
  • A hybrid FE-ray approach generated 25,625 simulated ultrasonic images to train a DL model estimating crack length and angle.The model was applied to experimental data, addressing limited real training data.
  • Regression of defect parameters remains underexploited, although it could support systematic evaluation of the criticality of small defects.Modelling data can help train models applied to experimental measurements when defect data are limited.

3.4. Property measurement

DL has been used to infer material properties from ultrasonic data, including porosity, grain size, and stiffness-related microstructure parameters. These studies combine learned models with simulated, augmented, or full-wavefield data for efficient inversion.

  • DL models infer porosity levels in additive-manufactured parts from ultrasonic signals and C-scan images.
  • 75,000 simulated and noise-augmented samples produced excellent correlation between predicted and ground-truth porosity levels.The DL method was more efficient than classical inversion requiring thousands of forward-model evaluations.
  • A CNN followed by fully connected layers estimated spatial grain-size distributions in metals from B-scan images.
  • Physics-informed neural networks inferred three stiffness-tensor elements and quantified microstructure properties in polycrystalline nickel from ultrasonic wavefield data.

3.5. Common challenges in the literature

Across ultrasonic NDE applications, limited training data, inconsistent performance evaluation, and insufficient uncertainty quantification remain shared challenges. Data augmentation, generative models, transfer learning, and domain adaptation are among the approaches used to address them.

  • A lack of training data is a shared challenge across DL models for NDE, addressed through data augmentation, generative models, and transfer learning.
  • Domain adaptation refines models trained in one domain using data from another, such as fine-tuning modelling-trained models with a small experimental subset.This approach uses physics-based models when experimental data are limited.
  • No universally accepted performance-quantification method makes it difficult to compare DL studies, even for the same application.
  • Uncertainty quantification for output variance has received little attention, and a global method covering data and model uncertainties remains unresolved.Techniques such as Monte Carlo dropout exist, but no definitive answer is established.

4. Levels of automation

The paper proposes a progression from human-led ultrasonic NDE to fully autonomous inspection and decision-making, using DL to support increasingly complex automation. Its review maps current contributions mainly to partial automation while identifying data, generality, uncertainty, reliability, and certification challenges.

  • Automation roadmap: A clear sequence of automation levels is proposed to categorize DL contributions and establish paths toward greater automation in NDE.The levels extend from classical human-operated procedures to fully autonomous NDE without human intervention.
  • Automation roadmap: At Levels 3 and 4, operational NDE is completely automated and can directly feed structural-integrity decisions such as accept/reject, repair, or remaining-useful-life estimation.These levels extend automation beyond inspection into data-driven structural-integrity decision-making.
  • Automation roadmap: DL is identified as an enabling factor for higher automation, potentially reducing operator input while handling increasingly complex inspection scenarios.The proposed progression ultimately allows decisions to be taken without human intervention.
  • Level 2: Partial automation: Level 2 systems still require improvement in generality, explainability, and adaptability beyond laboratory environments.Robustness across varying conditions may require physics-based training data, long-term data, domain adaptation, or unsupervised learning validated across laboratory and industrial trials.
  • Level 2: Partial automation: Level 2 also raises challenges in propagating uncertainty between independently designed models as information passes through the autonomous system.This is identified as an incipient area relevant to complex structures in sectors such as aerospace and wind energy.
  • Level 4: Full automation: Level 4 remains aspirational and would require extremely high accuracy across possible damage, material, environmental, and experimental conditions, alongside stricter certification and auditability.The system would also need clearly labelled outputs and physics-informed data for autonomous maintenance and manufacturing decisions.
  • Level 4: Full automation: Fully autonomous systems require sufficiently high confidence and explicit reliability-quantification methods that account for diverse uncertainties.Retaining inspectors' and regulators' understanding of the underlying physics and DL operation is also identified as necessary.
  • Mapping contributions: The review maps all DL-related contributions to Levels 1 and 2, with most work targeting partial automation through detection, characterisation, and property measurement.Higher automation levels remain aspirational, while methodology development at Levels 1 and 2 is still described as being in its infancy.

5. Basic axioms for DL-based ultrasonic NDE

The paper proposes six axioms to make DL-based ultrasonic NDE methods measurable, bounded, uncertainty-aware, parameterized, self-aware, and traceable. The review finds heterogeneous adoption of the first three axioms and identifies substantial gaps in parameters, self-awareness, and traceability for industrial deployment.

  • Proposed axioms: Six axioms define requirements for DL-based NDE, spanning operational scope, performance, uncertainty, adjustable parameters, self-awareness, and traceability.The axioms are intended to support realistic and industrially applicable inspection procedures.
  • Axioms 1–3: DL models must be trained within a predefined domain of operation rather than expected to work across all conditions, structures, and techniques.The scope should reflect the NDE modality, inspected structures and materials, and expected environmental changes.
  • Axioms 1–3: Performance must be quantified across the operating domain using task-appropriate measures, including probability of detection, probability of false alarm, or ROC curves for defect detection.Accuracy or error levels alone may not evaluate a model in an inspection environment.
  • Axioms 1–3: Uncertainties in the model, data, and hypotheses must be quantified to provide confidence information for autonomous operation.Suggested approaches include Monte Carlo dropout, Markov chain Monte Carlo, variational inference, and Bayesian active learning.
  • Axioms 4–6: Adjustable model parameters must be specified because preprocessing choices such as scaling or delaying can influence trained-model performance.Parametric studies can quantify output variation caused by influential inspection parameters.
  • Axioms 4–6: Higher-level autonomous systems need self-awareness to identify out-of-domain inputs and traceability to connect outputs with explanatory labels, metadata, and source data.Epistemic uncertainty can help detect out-of-domain data, while APIs and logs can support traceability.
  • Evidence from reviewed studies: Most reviewed studies address domain description and performance, with some uncertainty analysis, but approaches to performance and uncertainty quantification lack consensus.Many studies report accuracy or error without directly evaluating inspection measures such as probability-of-detection curves or ROC curves.
  • Remaining gaps: Industrial applicability remains constrained by incomplete treatment of influential parameters, standardized out-of-domain procedures, and traceability of assembled models.These gaps correspond to Axioms 4–6 and require responses from the scientific community, software engineering, systems integration, and operational procedures.

6. Conclusions and future directions

The review finds that DL is being used across ultrasonic NDE tasks, but development practices remain heterogeneous and current research is concentrated on operator assistance and partial automation. It proposes a roadmap, axioms, and research directions centered on focused signal processing, image interpretation, uncertainty quantification, and autonomous self-awareness.

  • Conclusions: DL has been applied to data processing, operator-independent damage characterisation, and repetitive defect detection, but heterogeneous development practices may limit industrial applicability.The reviewed applications include tasks that were slow or impractical with classical approaches.
  • Conclusions: Current research is concentrated around Levels 1 and 2 of the proposed roadmap: operator assistance and partial automation.The roadmap ranges from fully operator-driven NDE to fully automated NDE and structural integrity assessment.
  • Automation roadmap: The roadmap sets a horizon for researchers and industry by defining expected human and autonomous-system obligations at successive automation levels.Its stated aim is to make objectives for advancing automation easier to set and standardize the automation journey.
  • Future opportunities: Focused denoising should identify defect-echo time of flight or separate overlapping echoes, rather than only suppressing raw-signal noise.The paper links cleaner, focused data with potentially improved probability of detection, fewer false alarms, and easier defect characterisation.
  • Future opportunities: DL super-resolution methods are being explored for more accurate ultrasonic images, with further development needed for materials having complex internal structures.The paper presents improved interpretability for NDE operators as a potential benefit.
  • Future opportunities: Real inspection use requires uncertainty quantification that distinguishes experimental measurement variability from irreducible model uncertainty.The paper identifies confidence intervals as critical for refining models toward accuracy and reliability.
  • Future opportunities: Autonomous systems must detect inputs outside their predefined operating domain and flag them for operator takeover or model retraining or redesign.Epistemic uncertainty is identified as one possible route to self-awareness.
  • Conclusions: The proposed axioms cover both model evaluation and implementation, including quantifiable performance, uncertainty quantification, and traceability.They were derived from the literature review with input from diverse industries.

CRediT authorship contribution statement

The authors divide contributions across conceptualization, methodology, analysis, visualization, writing, resources, project administration, review, and supervision.

  • Authorship contributions: Sergio Cantero-Chinchilla led conceptualization, methodology, formal analysis, visualization, and original-draft writing.His contribution statement also lists writing, original draft.
  • Authorship contributions: Paul D. Wilcox contributed conceptualization, writing and review, resources, and project administration.The statement specifies writing—review and editing.
  • Authorship contributions: Anthony J. Croxford contributed conceptualization, writing and review, and supervision.The statement specifies writing—review and editing.
Loading 2112.06650v2…