Source-linked AI summary
1D Convolutional Neural Networks and Applications: A Survey
Serkan Kiranyaz, Onur Avci, Osama Abdeljaber, Turker Ince, Moncef Gabbouj, Daniel J. Inman
TL;DR
The paper addresses limitations of deep 2D CNNs for scarce or application-specific 1D-signal data. It reviews 1D CNN architectures and engineering applications, reporting state-of-the-art performance and feasible real-time, low-cost implementation.
Problem
Deep 2D CNNs may not be viable for numerous 1D-signal applications when training data is scarce or application-specific.
Method
The paper comprehensively reviews 1D CNN architecture, principles, engineering applications, benchmark datasets, and software.
Results
1D CNNs achieve close to 100% accuracy for detecting motor bearing anomalies and state-of-the-art performance across several engineering applications.
Takeaways & Limitations
Compact 1D CNNs can support real-time anomaly detection and relatively low-cost hardware implementation through simple one-dimensional operations.
Takeaways & Limitations
Deep 1D CNN configurations share the common drawbacks of their 2D counterparts, requiring techniques such as data augmentation and batch normalization to improve generalization.
Abstract
from arXiv · showhide
During the last decade, Convolutional Neural Networks (CNNs) have become the de facto standard for various Computer Vision and Machine Learning operations. CNNs are feed-forward Artificial Neural Networks (ANNs) with alternating convolutional and subsampling layers. Deep 2D CNNs with many hidden layers and millions of parameters have the ability to learn complex objects and patterns providing that they can be trained on a massive size visual database with ground-truth labels. With a proper training, this unique ability makes them the primary tool for various engineering applications for 2D signals such as images and video frames. Yet, this may not be a viable option in numerous applications over 1D signals especially when the training data is scarce or application-specific. To address this issue, 1D CNNs have recently been proposed and immediately achieved the state-of-the-art performance levels in several applications such as personalized biomedical data classification and early diagnosis, structural health monitoring, anomaly detection and identification in power electronics and motor-fault detection. Another major advantage is that a real-time and low-cost hardware implementation is feasible due to the simple and compact configuration of 1D CNNs that perform only 1D convolutions (scalar multiplications and additions). This paper presents a comprehensive review of the general architecture and principals of 1D CNNs along with their major engineering applications, especially focused on the recent progress in this field. Their state-of-the-art performance is highlighted concluding with their unique properties. The benchmark datasets and the principal 1D CNN software used in those applications are also publically shared in a dedicated website.
1 Introduction
CNNs evolved from biologically inspired and early neural-network models into dominant deep-learning tools, while 1D CNNs address signal applications where deep 2D CNN requirements are impractical.
- Origins: Biological-neuron research and the Neocognitron established hierarchical processing ideas that influenced CNN development.Hubel and Wiesel described receptive fields and hierarchical visual processing; Fukushima and Miyake later proposed the self-organized Neocognitron.
- CNN development: LeCun’s back-propagation-trained LeNet and its MNIST classification performance helped initiate the modern CNN era.LeNet matured in 1998, when its classification power was demonstrated on the benchmark MNIST handwritten-number database.
- Deep 2D CNNs: 16.4% error rate made AlexNet about 10% better than the second-ranked traditional machine-learning method on ImageNet in 2012.AlexNet was an eight-layer CNN with five convolutional-pooling layers and three fully connected layers.
- Deep 2D CNNs: 6.7% ImageNet error rate was achieved by GoogLeNet in 2014 through greater depth and width while maintaining a constant computational budget.GoogLeNet used 22 convolutional layers and nearly halved the best preceding error rate.
- CNN advantages and constraints: CNNs jointly learn feature extraction and classification from raw data instead of relying on fixed handcrafted features and preprocessing.This integrated learning approach made CNNs attractive for complicated engineering applications, although deep CNN training is computationally demanding.
- Motivation for 1D CNNs: Deep 1D CNNs can require extensive training data and additional techniques, limiting their practicality when application-specific datasets are scarce.One cited study used more than 96% of its total data to train a deep 1D CNN and applied augmentation, normalization, dropout, and majority voting.
2 Overview of Convolutional Neural Networks
CNNs combine convolutional and pooling operations with fully connected layers to learn features and classifications from input data. The 1D variant replaces image-oriented matrix operations with compact array-based processing for variable, data-limited signals.
- 2D Convolutional Neural Networks: Deep CNNs gained popularity because they combine feature extraction and classification in one trainable system.They learn task-specific features directly from raw input rather than relying on fixed hand-crafted features.
- 2D Convolutional Neural Networks: A conventional CNN applies convolution and pooling layers before fully connected layers produce the classification output.The sample architecture processes a 24×24 grayscale image through two convolution and pooling stages, followed by fully connected and output layers.
- 2D Convolutional Neural Networks: LeNet and AlexNet share convolutional and pooling principles, but AlexNet uses a substantially deeper architecture with millions of parameters.LeNet has two interleaved convolutional and pooling layers, whereas AlexNet has five convolutional and three max-pooling layers.
- 2.2 1D Convolutional Neural Networks: Compared with 2D CNNs, 1D CNNs use simple array operations and can achieve challenging signal-processing performance with relatively shallow architectures.The paper links this compactness to lower computational complexity and suitability for applications with limited labeled data and high signal variation.
- 2.2 1D Convolutional Neural Networks: 1D CNNs replace 2D kernels and feature maps with 1D arrays, enabling convolutional processing of raw 1D signals before MLP-based classification.Their CNN layers perform 1D convolution and subsampling, while fully connected layers perform the classification task.
- 2.3 Forward- and Back-Propagation in CNN-layers: The 1D CNN topology can adapt to variations in input-layer dimension by tuning the subsampling factor of the output CNN layer.Forward propagation uses 1D convolutions, activation, and subsampling; back-propagation is formulated for the CNN layers.
3 Applications of 1D CNNs
Compact 1D CNNs are applied across biomedical monitoring, structural damage detection, rotating-machine condition monitoring, and power-electronics fault diagnosis. The reviewed applications emphasize reduced feature-engineering demands, limited-data training, real-time operation, and high reported detection performance.
- 1D CNNs have been preferred over 2D counterparts in engineering applications because they combine adaptive feature extraction and classification for one-dimensional signals.The survey identifies applications spanning ECG monitoring, civil structures, rotating machinery, and power electronics.
- 3.1 Real-time Electrocardiogram (ECG) Monitoring: Personalized ECG monitoring addresses the difficulty of detecting arrhythmias in healthy people without abnormal training beats.Conventional patient-specific classifiers require training samples containing both normal and abnormal beats, whereas the reviewed personalized approach targets anomalies as they appear.
- 3.2 Vibration-Based Structural Damage Detection in Civil Infrastructure: 1D CNN vibration methods detect structural damage directly from raw signals without handcrafted features and can require substantially less labeled data.The survey contrasts these methods with conventional approaches that depend on computationally costly feature extraction and reports successful testing on the SHM Benchmark Problem.
- 3.2 Vibration-Based Structural Damage Detection in Civil Infrastructure: A local 1D CNN was trained for each sensor location on vibration responses from 31 damage scenarios in a large-scale laboratory structure.The method processed local sensor data and was tested across numerous single- and double-damage cases.
- 3.3 Condition Monitoring in Rotating Mechanical/Aerospace Machine Parts: Compact 1D CNNs support rotating-machine anomaly monitoring, with one motor-bearing study reporting close to 100% accuracy despite visually subtle fault signatures.The reviewed method uses layered sub-band decomposition in hidden CNN layers and compares its ROC curves with conventional methods.
- 3.4 Fault Detection in Modular Multilevel Converters (MMC): For modular multilevel converters, the reviewed system reports practically 100% fault detection and identification, less than 0.1 s delay, low computational complexity, and no false alarms.The system is also described as robust to parameter and fault-time variations and scalable through parallel dedicated 1D CNNs.
4 Computational Complexity Analysis of 1D-CNNs
The analysis computes 1D-CNN forward- and backward-propagation operations layer by layer, showing that complexity is driven mainly by interlayer connections and convolution sizes. Across the reviewed applications, 1D CNNs achieve minimal computational complexity and very fast fault detection.
- Forward propagation: Forward-pass complexity is obtained by counting multiplications and additions at each layer and summing them across the CNN.The analysis ignores subsampling and bias additions where their computational costs are negligible.
- Layer-wise complexity: Each 1D-CNN layer’s operation count depends on the number of connections between adjacent layers and the convolution dimensions.The number of connections is the product of the neuron counts in consecutive layers, while each connection performs a linear convolution.
- Backward propagation: During backpropagation, two convolutions contribute additional multiplication and addition costs for error propagation and weight updates.The analysis identifies the second convolution as operating between the current-layer output and related quantities, with its costs included in the total BP count.
- Complexity comparison: Multiplications dominate additions, especially for high kernel sizes, and both operation complexities scale with the total number of connections between consecutive layers.The corresponding MLP computation is comparatively negligible because each connection uses only a scalar multiplication and addition.
- Application-level performance: 45-times faster than real-time speed was achieved for fault detection on an ordinary computer, while Figure 17 compares average motor-fault classification times against six competing algorithms.The figure reports execution times in milliseconds for the 1D CNN and six competing methods.
5 Conclusions
The paper situates CNNs within the broader history of machine learning while acknowledging that modern deep neural networks remain simplified models of biological neural systems. Recent work has begun exploring more generalized neuron and network models to address this deficiency.
- Historical context: CNNs emerged after decades of development in artificial-neuron and machine-learning models, attracting substantial attention as a prominent machine-learning architecture.The historical discussion begins with the McCulloch-Pitts artificial-neuron model introduced in 1943.
- Biological scope: Modern deep neural networks are described as highly reductive compared with the diverse electrochemical and cellular processes of biological brains.The cited neuroscience perspective emphasizes the complexity of neuronal types, synapses, and non-neuron brain cells.
- Emerging directions: Recent studies have proposed generalized neuron and network models intended to address limitations in modern deep or compact artificial neural networks.These efforts are presented as responses to the deficiency identified in biological fidelity.