Source-linked AI summary
Multivariate LSTM-FCNs for Time Series Classification
Fazle Karim, Somshubra Majumdar, Houshang Darabi, Samuel Harford
TL;DR
Multivariate time series classification is important across complex tasks, but existing approaches can require difficult feature extraction or other preprocessing. The paper converts LSTM-FCN and ALSTM-FCN into multivariate models by adding squeeze-and-excitation blocks. The resulting models achieve state-of-the-art performance on most tested datasets while remaining efficient and minimally preprocessed.
Problem
Existing multivariate time-series classifiers use varied representations and may rely heavily on difficult feature extraction, motivating models with minimal preprocessing.
Method
The paper adapts LSTM-FCN and ALSTM-FCN to multivariate inputs by augmenting their fully convolutional blocks with squeeze-and-excitation blocks.
Results
The proposed models attain state-of-the-art results on 28 of 35 tested datasets and significantly outperform the original LSTM-FCN and ALSTM-FCN models.
Takeaways & Limitations
The models require minimal preprocessing and are small and efficient enough for real-time and embedded-system deployment.
Abstract
from arXiv · showhide
Over the past decade, multivariate time series classification has received great attention. We propose transforming the existing univariate time series classification models, the Long Short Term Memory Fully Convolutional Network (LSTM-FCN) and Attention LSTM-FCN (ALSTM-FCN), into a multivariate time series classification model by augmenting the fully convolutional block with a squeeze-and-excitation block to further improve accuracy. Our proposed models outperform most state-of-the-art models while requiring minimum preprocessing. The proposed models work efficiently on various complex multivariate time series classification tasks such as activity recognition or action recognition. Furthermore, the proposed models are highly efficient at test time and small enough to deploy on memory constrained systems.
1. Introduction
Multivariate time series classification supports applications across healthcare, phoneme, activity, object, and action recognition, but existing approaches differ in feature-extraction demands and modeling strategies. This paper proposes two deep learning models requiring minimal preprocessing and evaluates them on 35 datasets.
- Multivariate time series classification is applied to healthcare, phoneme, activity, object, and action recognition.
- Distance-based methods, including k-nearest neighbors with Dynamic Time Warping, have been successful for multivariate classification.
- Feature-based algorithms rely heavily on extracted features, but intrinsic time-series features are difficult to capture and extraction is arduous.
- Traditional models such as naive logistic and Fisher kernel learning show strong performance across varied time-series classification problems.
- Symbolic and learned-pattern approaches use dimensional reduction, concatenation, or tree-derived representations to classify multivariate time series.
- Deep learning approaches include MC-DCNN, which detects latent features from each variable before classification with an MLP.
- The paper proposes two deep learning models with minimal preprocessing and tests them on 35 datasets.
2. Background Works
The background introduces recurrent, LSTM, attention, and squeeze-and-excitation mechanisms used to model temporal dependencies and recalibrate feature maps. These components motivate the paper’s multivariate architecture.
- Recurrent Neural Networks: RNNs model temporal behavior through direct connections between network layers and update a hidden vector at each time step.
- Long Short-Term Memory RNNs: LSTMs address vanishing gradients with gating functions and maintain hidden and memory vectors for state updates and outputs.
- Long Short-Term Memory RNNs: The LSTM computation uses input, forget, output, and cell-state gates parameterized by recurrent and projection weight matrices.
- Attention Mechanism: Attention mechanisms target long-term dependencies by conditioning a context vector on input-sequence annotations and learning alignment weights.
- Squeeze-and-Excitation Block: Squeeze-and-excitation blocks model channel interdependencies by applying squeeze and excitation operations to feature maps.
- Squeeze-and-Excitation Block: For temporal data, squeezing computes channel-wise statistics by averaging transformation outputs across the temporal dimension.
- Squeeze-and-Excitation Block: The excitation mechanism uses sigmoid gating, ReLU activation, and reduction and expansion layers to capture channel-wise dependencies while limiting complexity.
- Squeeze-and-Excitation Block: The block finally rescales its output feature maps after channel-wise excitation.
3. Multivariate LSTM Fully Convolutional Network
MLSTM-FCN and MALSTM-FCN adapt univariate LSTM-FCN architectures for multivariate inputs by combining convolutional feature extraction, an LSTM path, and squeeze-and-excitation recalibration. The architecture also uses dimension shuffling to process multivariate temporal information efficiently.
- Multivariate adaptation: MLSTM-FCN and MALSTM-FCN convert LSTM-FCN and ALSTM-FCN into multivariate models by accepting M variables per time step.The input dataset is represented as a tensor with N samples, Q time steps, and M variables.
- Network architecture: The proposed architectures combine a three-block temporal convolutional feature extractor with an LSTM or Attention LSTM block.The convolutional blocks use 128, 256, and 128 filters with kernel sizes 8, 5, and 3, respectively.
- Squeeze-and-excitation: With r = 16, the squeeze-and-excitation mechanism increases overall model size by just 3-10%.Because the convolutional blocks are consistent across models, the additional parameter count is computed as P = 2/16 × (128^2 + 256^2) = 10240.
- Squeeze-and-excitation: Squeeze-and-excitation blocks adaptively recalibrate convolutional feature maps, adding learned self-attention to inter-correlations among variables.The blocks are inserted into the fully convolutional path and use reduction ratio r = 16.
- Dimension shuffle: Dimension shuffling transposes the temporal dimension so the LSTM processes M variable histories as time steps, reducing training and inference computation without losing classification accuracy.Without shuffling, the LSTM processes Q time steps with M variables; with shuffling, it processes M time steps with Q variables.
4. Experiments
The experiments evaluate MLSTM-FCN and MALSTM-FCN on 35 datasets using standardized preprocessing, multiple performance metrics, and comparisons with existing models. Both proposed models outperform state-of-the-art methods on most datasets, while dimension shuffle substantially reduces processing time without changing accuracy statistically.
- Evaluation metrics: Accuracy, arithmetic rank, geometric rank, Wilcoxon signed-rank tests, and mean per class error assess model performance.Arithmetic and geometric rank summarize model ranks across datasets, while mean per class error averages class-wise errors.
- Datasets: 35 datasets from benchmark and UCI repositories are used, with predefined or 50:50 train-test splits depending on the dataset.The datasets cover various classification tasks and are normalized to zero mean and unit standard deviation, with zero padding for variable lengths.
- Results: MLSTM-FCN and MALSTM-FCN outperform state-of-the-art models on 28 and 27 of 35 datasets, respectively.Activity and Action 3d require strided temporal convolution before the LSTM branch to reduce memory use; Activity is among the few datasets without improvement over SOTA.
- Results: MLSTM-FCN and MALSTM-FCN achieve arithmetic ranks of 3.29 and 3.17, and geometric ranks of 2.58 and 2.42, respectively.A critical difference diagram compares arithmetic mean ranks across all 35 datasets.
- Statistical analysis: The Wilcoxon analysis finds both proposed models significantly better than LSTM-FCN and ALSTM-FCN, while their performance is statistically indistinguishable.The adjusted significance level is 0.0028, and the authors attribute the improvement to modeling inter-dependencies between variables with squeeze-and-excitation blocks.
- Ablation tests: Dimension shuffle reduces processing time by 59 percent while maintaining statistically equivalent classification accuracy.Processing all datasets falls from approximately 32 hours without dimension shuffle to 13 hours with it; MPCE is 4.86 versus 4.21, respectively.
5. Conclusion & Future Work
The proposed models achieve strong results across the evaluated datasets while requiring minimal preprocessing and feature extraction. They are also positioned for efficient deployment in real-time and embedded systems.
- The proposed models require minimal preprocessing and feature extraction.
- The models are small and efficient enough for quick deployment in real-time and embedded systems.
Appendix A. Variable Definitions
Appendix A includes Table A.5, which presents definitions for all variables.
- Table A.5 is titled “Definition of all variables.”
- The table serves as a variable-definition reference in Appendix A.
- Appendix A organizes the variable definitions under Table A.5.