PaperlayerView exact revision on arXiv

Source-linked AI summary

1D Convolutional Neural Networks and Applications: A Survey

Serkan Kiranyaz, Onur Avci, Osama Abdeljaber, Turker Ince, Moncef Gabbouj, Daniel J. Inman

arXiv:1905.03554v1eess.SPcs.AI

TL;DR

Conventional vibration-based damage-detection methods rely on handcrafted features that may not suit all structures or damage types and require substantial computational effort. This survey reviews 1D CNN architecture and engineering applications, reporting that shallow networks can learn challenging 1D-signal tasks and that motor-anomaly detection can achieve close to 100% accuracy.

  • Problem

    Conventional vibration-based damage-detection methods rely on handcrafted feature and classifier combinations that may not suit all structures or damage types and require substantial computational effort.

    Source: §3.2 · p. 11

  • Method

    The paper describes 1D CNNs processing raw signals with one-dimensional kernels and feature maps, learning features for classification.

    Source: §2.2 · p. 7

  • Results

    Shallow 1D CNNs can learn challenging 1D-signal tasks, while motor-anomaly detection with a compact 1D CNN reached close to 100% accuracy.

    Source: §2.2 · p. 6, §3.3 · p. 14

  • Takeaways & Limitations

    Simple array operations make 1D CNNs less computationally complex than 2D CNNs, and compact models can be trained on standard computers using CPUs.

    Source: §2.2 · p. 6

    Deep 1D CNN configurations share drawbacks with their 2D counterparts, and the cited bearing-fault approach used several tricks to improve generalization.

    Source: §1 · p. 4

Abstract

from arXiv · show

During the last decade, Convolutional Neural Networks (CNNs) have become the de facto standard for various Computer Vision and Machine Learning operations. CNNs are feed-forward Artificial Neural Networks (ANNs) with alternating convolutional and subsampling layers. Deep 2D CNNs with many hidden layers and millions of parameters have the ability to learn complex objects and patterns providing that they can be trained on a massive size visual database with ground-truth labels. With a proper training, this unique ability makes them the primary tool for various engineering applications for 2D signals such as images and video frames. Yet, this may not be a viable option in numerous applications over 1D signals especially when the training data is scarce or application-specific. To address this issue, 1D CNNs have recently been proposed and immediately achieved the state-of-the-art performance levels in several applications such as personalized biomedical data classification and early diagnosis, structural health monitoring, anomaly detection and identification in power electronics and motor-fault detection. Another major advantage is that a real-time and low-cost hardware implementation is feasible due to the simple and compact configuration of 1D CNNs that perform only 1D convolutions (scalar multiplications and additions). This paper presents a comprehensive review of the general architecture and principals of 1D CNNs along with their major engineering applications, especially focused on the recent progress in this field. Their state-of-the-art performance is highlighted concluding with their unique properties. The benchmark datasets and the principal 1D CNN software used in those applications are also publically shared in a dedicated website.

1 Introduction

Deep 2D CNNs became powerful but require extensive labeled data and computation, making them a poor fit for many practical 1D signal applications. This survey focuses on compact 1D CNNs for data-scarce or application-specific engineering problems and shares related benchmark datasets and software.

1 Introduction

  • The introduction traces CNN foundations from artificial-neuron models, through the visual-cortex-inspired Neocognitron, to LeNet trained with back-propagation.Artificial neurons model biological processing as a weighted sum followed by an activation function; the Neocognitron introduced hierarchical visual pattern recognition.Source: §1 · p. 2
  • AlexNet achieved a 16.4% error rate on ImageNet, about 10% lower than the second-top traditional SVM method using SIFT and LBP features.Its success was enabled by training on a massive dataset with parallelized computation on GPUs.Source: §1 · p. 3
  • ZFnet reduced ImageNet error to 11.7%, while 22-convolutional-layer GoogLeNet reached 6.7% in 2014.GoogLeNet increased network depth and width while keeping computational budget constant, and used an ensemble of six CNNs.Source: §1 · p. 3
  • CNNs learn task-optimized features from raw data, but 2D models require signal-to-image conversion and substantial computation and labeled data, limiting many 1D applications.Examples include reshaping vibration signals or converting ECG beats to spectrogram images; such CNNs are unsuitable for many real-time, low-power settings.Source: §1 · p. 3
  • Compact adaptive 1D CNNs were proposed for patient-specific ECG data and have been applied to arrhythmia detection, structural monitoring, circuitry monitoring, and fault diagnosis.A deep bearing-fault model used data augmentation and other techniques, with more than 96% of the total data used for training.Source: §1 · pp. 3–4
  • The survey centers on compact 1D CNNs for scarce labeled data or application-specific solutions, and makes benchmark datasets and software publicly available.It reviews 1D CNN architecture and back-propagation, engineering applications, computational complexity, and future directions.Source: §1 · p. 4

2 Overview of Convolutional Neural Networks

CNNs use convolutional and pooling operations to learn features from data, while 1D CNNs adapt this approach to signals with compact architectures. The section outlines their computational advantages, operation, and training process.

2 Overview of Convolutional Neural Networks

  • CNNs learn features directly from raw inputs, use sparse connections and tied weights for computational efficiency, tolerate small input transformations, and accommodate different input sizes.Source: §2 · p. 4
  • A conventional CNN applies convolution and pooling to feature maps before passing outputs to fully connected layers for classification.Convolution uses a kernel over the image, while pooling subsamples feature maps; the final fully connected output layer corresponds to the classes.Source: §2 · pp. 4–5
  • LeNet and AlexNet share convolutional-layer weight sharing and limited connectivity, while AlexNet's deeper architecture contains millions of parameters.Source: §2 · p. 5

2.2 1D Convolutional Neural Networks

  • Compact 1D CNNs can perform well on limited labeled data with high signal variation, using simpler, lower-complexity operations suited to CPU training and real-time, low-cost applications.The passage contrasts shallow 1D architectures with the deeper networks often needed for challenging 1D tasks and notes their suitability for mobile and handheld devices.Source: §2.2 · p. 6
  • 1D CNN layers apply convolutions, activation, and subsampling to raw signals, learning features for MLP classification in a jointly optimized process with low computational complexity.Their kernels and feature maps are 1D arrays; the topology can also adapt to variations in input dimension by tuning the output layer's subsampling factor.Source: §2.2 · p. 7

2.3 Forward- and Back-Propagation in CNN-layers

  • 1D CNN training initializes weights and biases, repeatedly forward-propagates signals, back-propagates output error, computes parameter sensitivities, and updates the parameters.Source: §2.3 · p. 9

3 Applications of 1D CNNs

The survey describes compact 1D CNNs across engineering applications, leveraging their ability to combine feature extraction and classification. The applications include cardiac monitoring, structural damage detection, rotating-machine condition monitoring, and converter fault detection.

3.1 Real-time Electrocardiogram (ECG) Monitoring

  • Patient-specific 1D CNNs classified ECG beats into five classes, achieving average accuracies of 99% for VEB and 97.6% for SVEB.The evaluation used 83,648 test beats from 44 records in the MIT/BIH arrhythmia database.Source: §3.1 · p. 10
  • Acc = 80.1% and FAR = 0.43% were achieved for personalized early arrhythmia detection without real abnormal beats in training.The average probability of missing all three consecutive abnormal beats was around 0.0079, making detection among the first three occurrences highly probable (> 99.2%).Source: §3.1 · p. 10

3.2 Vibration-Based Structural Damage Detection in Civil Infrastructure

  • 1D CNNs detected and located structural damage directly from raw vibration signals; loosened-bolt cases were detected without misses or false alarms at 45x real-time speed.The laboratory study trained an individual CNN for each sensor location using vibration data from 31 damage scenarios.Source: §3.2 · pp. 11–12
  • A 1D CNN approach reduced the effort and labeled data needed for structural-damage training and was tested on Experimental Phase II of the SHM Benchmark Problem.The preceding methods required many measurement sessions, especially for large civil structures.Source: §3.2 · p. 12

3.3 Condition Monitoring in Rotating Mechanical/Aerospace Machine Parts

  • Compact 1D CNNs detected motor-current bearing-fault anomalies with close to 100% accuracy through layered sub-band decomposition.The faults cause slight frequency variations that are difficult to detect visually, even with spectral analysis.Source: §3.3 · pp. 13–14

3.4 Fault Detection in Modular Multilevel Converters (MMC)

  • MMC 1D CNNs achieved practically 100% fault detection and identification accuracy, <0.1s delay, robustness to parameter and fault-time variation, and no false alarms.The approach can scale to converters with hundreds or thousands of cells by monitoring groups with dedicated CNNs in parallel.Source: §3.4 · p. 15

4 Computational Complexity Analysis of 1D-CNNs

The section estimates 1D-CNN computational cost by counting multiplications and additions in forward and backward passes, finding that additions are generally insignificant and that operations scale with connections between layers. It also reports low complexity relative to competing methods in surveyed applications.

4 Computational Complexity Analysis of 1D-CNNs

  • Forward-pass cost is calculated by counting multiplications and additions for each layer’s linear convolutions, while ignoring subsampling, boundary conditions, and bias additions.The analysis first counts operations per layer and then sums them across the CNN.Source: §4 · p. 16
  • Backpropagation performs two convolutions per iteration: one involving next-layer delta errors and another involving the current-layer output.The analysis counts these operations across connections between consecutive layers, ignoring boundary conditions.Source: §4 · p. 16
  • Across forward and backward passes, additions are insignificant compared with multiplications, especially for larger kernels, and operation complexity is proportional to interlayer connection counts.The connection count is the product of the numbers of neurons in consecutive layers.Source: §4 · p. 16
  • MLP computational cost is negligible in the described implementation because each connection requires only a scalar weight multiplication and addition.The passage characterizes MLP complexity as well known and cites as an example reference.Source: §4 · p. 17
  • Across surveyed applications, 1D CNNs achieve minimal computational complexity versus conventional methods; reports fault detection at 45-times real-time speed, and Figure 17 compares motor-fault classification times.Figure 17 shows average execution times for the 1D CNN method [55] and six competing algorithms [75–78]; the supplied passage gives no specific times or winner.Source: §4 · p. 17, Figure 17 · p. 17

5 Conclusions

The conclusion presents compact 1D CNNs as high-performing options for data-scarce applications requiring low-cost, real-time implementation, and points to emerging Operational Neural Networks as a possible next step for complex 1D signals.

5 Conclusions

  • CNNs learn directly from raw data, removing the need for handcrafted preprocessing and feature extraction, while deep CNNs can learn complex patterns from massive datasets.The conclusion frames this capability as a major development in machine learning and AI.Source: §5 · p. 17
  • Compact 1D CNNs can surpass conventional approaches and offer state-of-the-art performance with low computational complexity for data-scarce, real-time applications.They avoid the extra time and resources of converting 1D signals into 2D form and are suited to mobile or hand-held devices with limited computation and battery life.Source: §5 · p. 17
  • Conventional CNNs and ANNs use homogeneous, linear neurons, while GOPs [16] [17] and ONNs [81] introduce heterogeneous, nonlinear-capable alternatives.ONNs can use varied operators to learn complex, multimodal functions with minimal network complexity and training data; the authors foresee emerging 1D ONNs providing compact, efficient solutions for complex signal patterns.Source: §5 · pp. 17–18

Related papers

Loading 1905.03554v1…