Source-linked AI summary
Imaging Time-Series to Improve Classification and Imputation
Zhiguang Wang, Tim Oates
TL;DR
The paper addresses how to represent time series so computer-vision techniques can classify and impute them. It encodes series as GASF/GADF and MTF images, applies Tiled CNNs and Denoised Auto-encoders, and reports strong classification results plus lower GASF-based imputation error. The analysis relates learned features to temporal dependencies preserved in the image representations.
Problem
The paper asks how time series can be encoded as images so machines can visually recognize, classify, and learn their structures and patterns.
Method
The authors encode time series with GASF/GADF and MTF images, use Tiled CNNs for classification, and train Denoised Auto-encoders on GASF images for imputation.
Results
The approaches achieve the best performance on 9 of 20 standard classification datasets, while GASF-based imputation MSE falls by 12.18%-48.02% versus raw data.
Takeaways & Limitations
GASF, GADF, and MTF provide complementary image channels, combining static and dynamic time-series information for classification.
Takeaways & Limitations
MTF generally has higher error rates than GAFs because its inverse mapping has greater uncertainty, and transition features alone may be insufficient for classification.
Abstract
from arXiv · showhide
Inspired by recent successes of deep learning in computer vision, we propose a novel framework for encoding time series as different types of images, namely, Gramian Angular Summation/Difference Fields (GASF/GADF) and Markov Transition Fields (MTF). This enables the use of techniques from computer vision for time series classification and imputation. We used Tiled Convolutional Neural Networks (tiled CNNs) on 20 standard datasets to learn high-level features from the individual and compound GASF-GADF-MTF images. Our approaches achieve highly competitive results when compared to nine of the current best time series classification approaches. Inspired by the bijection property of GASF on 0/1 rescaled data, we train Denoised Auto-encoders (DA) on the GASF images of four standard and one synthesized compound dataset. The imputation MSE on test data is reduced by 12.18%-48.02% when compared to using the raw data. An analysis of the features and weights learned via tiled CNNs and DAs explains why the approaches work.
1 Introduction
The paper proposes encoding time series as images so computer-vision methods can recognize, classify, and learn temporal structures. It introduces GASF/GADF and MTF representations and evaluates Tiled CNNs for classification and Denoised Auto-encoders for imputation.
- The framework encodes time series as images to support visual recognition, classification, and learning of structures and patterns.
- GASF/GADF and MTF are introduced as three image representations for time series.
- Tiled CNNs classify individual and compound image representations across 20 standard datasets.
- The approach achieves the best performance on 9 of 20 standard datasets against 9 previous and current best classification methods.
- Denoised Auto-encoders trained on GASF images reduce test-data imputation MSE by 12.18%-48.02% compared with raw data.
2 Imaging Time Series
The paper encodes time series as images using Gramian Angular Fields and Markov Transition Fields, preserving temporal structure for visual feature learning. GAFs use polar coordinates and angular relationships, while MTFs sequentially represent transition probabilities across quantile bins.
- Gramian Angular Field: GAFs represent rescaled time series in polar coordinates, encoding values as angular cosines and timestamps as radii.The mapping is bijective because cosine is monotonic over the relevant angular interval, and polar coordinates preserve absolute temporal relations.
- Gramian Angular Field: GASF and GADF use trigonometric sums or differences between time points to encode temporal correlations.The GAF matrix preserves temporal order from top-left to bottom-right, and its main diagonal contains original value or angular information.
- Gramian Angular Field: GAF matrices preserve dependencies across time intervals, but their n × n size motivates smoothing and dimensionality reduction through Piecewise Aggregation Approximation.PAA smooths the series while preserving trends before GAF construction.
- Markov Transition Field: The MTF extends a Markov transition matrix by preserving temporal dependency, addressing information loss caused by representing transitions without time ordering.The MTF encodes multi-span transition probabilities, with entries grouped by temporal intervals.
- Markov Transition Field: MTF construction quantizes a time series into Q quantile bins, forms a Markov transition matrix, and spreads its probabilities across temporal positions.Each MTF entry represents the transition probability between the bins containing observations at two time steps.
- Image-based learning: Tiled CNNs learn invariant, overcomplete image representations through multiple feature maps, tile-controlled weight sharing, and unsupervised TICA pretraining.The illustrated network stacks two Tiled CNNs and uses receptive fields, TICA pooling, and a final linear SVM.
- Markov Transition Field: Non-overlapping m × m patches average MTF pixels to aggregate transition probabilities and make image size and computation more manageable.The aggregation combines transition probabilities within subsequences of length m.
3 Classify Time Series Using GAF/MTF with Tiled CNNs
The study evaluates Tiled CNN classification from GAF, MTF, and compound image representations across 20 time-series datasets, comparing error rates with established methods. Compound GASF-GADF-MTF images combine static and dynamic information, while MTF has a broader inverse-image uncertainty that limits classification and recovery.
- Experimental setup: Tiled CNNs classify single GASF, GADF, and MTF images, as well as compound GASF-GADF-MTF images, across 20 datasets.The datasets span medicine, entomology, engineering, astronomy, signal processing, and other domains.
- Model: Tiled CNNs learn invariant, overcomplete features using tiled weight sharing and unsupervised Topographic ICA pretraining.Tile size controls the distance over which weights are shared, while multiple feature maps support overcomplete representations.
- Experimental setup: The experiments tune image size, quantile size, and SVM penalty, selecting the configuration with the lowest training error before test evaluation.Image sizes range from 16 to 48, quantile sizes from 8 to 64, and the SVM penalty is selected by 5-fold cross-validation.
- Limitations: MTF generally has higher error rates than GAFs because its inverse mapping has greater uncertainty, despite encoding transition dynamics.GAF mappings are bijective for 0/1-rescaled data, whereas MTF has a much larger inverse-image space.
- Representation: Compound GASF-GADF-MTF images combine GAF static information with MTF transition dynamics as three orthogonal image channels.The compound representation is intended to combine static and dynamic statistics embedded in the raw time series.
4 Image Recovery on GASF for Time Series Imputation with Denoised Auto-encoder
The paper reconstructs missing time-series values by recovering corrupted GASF images with a denoised auto-encoder, leveraging the bijection of GASF for 0/1-rescaled data. GASF-based imputation is more stable than raw-data imputation despite slightly higher full-sequence error.
- Image recovery: The main diagonal of GASF images enables precise recovery of 0/1-rescaled time series because the mapping is bijective.This supports predicting missing values by recovering broken GASF images.
- Image recovery: The denoising auto-encoder is trained on GASF images generated from time series corrupted with randomly masked points.Salt-and-pepper noise is added before transformation during training, while broken GASF images are reconstructed by a sigmoid hidden-layer model.
- Results: Imputation MSE is reduced by 12.18%-48.02% with GASF compared to raw time series.The comparison concerns MSE on only the unknown points.
- Results: GASF imputation has slightly higher full MSE but a smaller gap between full MSE and imputation MSE than raw-data imputation.The authors report that interpolation with GASF has more stable performance than interpolation on raw data.
- Interpretation: GASF preserves temporal and spatial dependencies, allowing the auto-encoder to use relations among missing points and other time-series positions.The transformation is described as a kernel-trick-like dimensionality increase that augments the data representation.
5 Analysis on Features and Weights Learned by Tiled CNNs and DA
The analysis interprets Tiled CNN features as multi-frequency representations of GASF and MTF images, while denoising auto-encoders learn image prototypes. Limited training data can produce noisy learned filters.
- Tiled CNN features: Tiled CNNs extract multi-frequency dependencies from GASF feature maps while preserving trends and representing details across subphases.The learned high-level maps are described as a multi-frequency approximator of the original curve.
- Tiled CNN features: The Tiled CNN color-patch operation acts as a learned two-dimensional temporal integration rather than a simple moving average.Different trained weights enhance receptive fields while using dependencies among different time intervals.
- Tiled CNN weights: The learned weight matrix satisfies WW^T = I, making effective use of local orthogonality.The analysis further states that this makes the system well-conditioned as the condition number of W approaches 1.
- DA features: Denoising auto-encoders learn different prototypes of GASF images rather than visual concepts such as edges or angles.These prototypes are learned from the GASF representation used for imputation.
- DA limitations: The 7 Misc dataset contains significantly noisy filters because its relatively small training set limits learning of distinct filters.The authors characterize patternless filters as functioning like a Gaussian-noise filter.
6 Conclusions and Future Work
The paper concludes that image representations combined with Tiled CNNs and denoising auto-encoders provide competitive classification and more stable GASF-based imputation. Future work targets streaming data, alternative architectures, richer generative models, and additional real-world tasks.
- Conclusions: The proposed pipeline converts time series into GASF, GADF, and MTF images and extracts multilevel features with Tiled CNNs and denoising auto-encoders.The pipeline addresses classification and imputation.
- Conclusions: The approach yields competitive classification results, while GASF imputation achieves better and more stable performance than raw-data imputation.The conclusion attributes the imputation comparison to denoising auto-encoders.
- Interpretation: The learned Tiled CNN features behave like a multi-frequency moving average that benefits from two-dimensional temporal dependencies preserved by the Gramian matrix.Denoising auto-encoder features form different prototypes that act as correlated bases for constructing raw images.
- Future work: Future work includes recurrent neural networks for streaming data and testing different deep-learning architectures on GAF and MTF images.The authors also propose deeper generative models for GAF images.
- Future work: The authors aim to apply the time-series models to real-world regression, imputation, and anomaly-detection tasks.