Source-linked AI summary
Deep Learning for Multivariate Time Series Imputation: A Survey
Jun Wang, Wenjie Du, Yiyuan Yang, Linglong Qian, Wei Cao, Keli Zhang, Wenjia Wang, Yuxuan Liang, Qingsong Wen
TL;DR
Missing values challenge multivariate time-series analysis, while existing surveys have limited coverage of deep learning-based imputation. This survey systematically reviews deep MTSI methods, proposes a taxonomy, summarizes toolkits, and discusses challenges and future directions. It emphasizes predictive and generative methods, the PyPOTS Ecosystem, and open issues including MNAR missingness, downstream integration, and scalability.
Problem
Missing values can reduce the accuracy and reliability of downstream multivariate time-series analysis, while prior surveys mainly emphasize statistical and traditional machine-learning imputation.
Method
The survey synthesizes deep learning MTSI methods through a taxonomy based on imputation uncertainty and neural network architecture, and reviews toolkits including PyPOTS.
Results
The survey presents a systematic review with a taxonomy of predictive and generative methods, an architecture overview, toolkit coverage, and discussion of large-model approaches.
Takeaways & Limitations
The survey identifies handling MNAR missingness, integrating imputation with downstream tasks, improving scalability, and exploring large-scale pre-trained and multimodal models as future directions.
Takeaways & Limitations
Deep learning imputation algorithms often have higher computational costs than statistical and machine-learning counterparts, creating scalability challenges as incomplete data volumes grow.
Abstract
from arXiv · showhide
Missing values are ubiquitous in multivariate time series (MTS) data, posing significant challenges for accurate analysis and downstream applications. In recent years, deep learning-based methods have successfully handled missing data by leveraging complex temporal dependencies and learned data distributions. In this survey, we provide a comprehensive summary of deep learning approaches for multivariate time series imputation (MTSI) tasks. We propose a novel taxonomy that categorizes existing methods based on two key perspectives: imputation uncertainty and neural network architecture. Furthermore, we summarize existing MTSI toolkits with a particular emphasis on the PyPOTS Ecosystem, which provides an integrated and standardized foundation for MTSI research. Finally, we discuss key challenges and future research directions, which give insight for further MTSI research. This survey aims to serve as a valuable resource for researchers and practitioners in the field of time series analysis and missing data imputation tasks.A well-maintained MTSI paper and tool list are available at https://github.com/WenjieDu/Awesome_Imputation.
1 Introduction
Multivariate time series often contain missing values that undermine downstream analysis, motivating deep learning-based imputation methods. This survey organizes the field through a taxonomy, reviews toolkits, and identifies future research directions.
- Motivation: Missing measurements arise from sensor failures, irregular sampling, privacy concerns, system downtime, and communication issues across application domains.Such gaps can significantly affect the accuracy and reliability of downstream analysis and decision-making.
- Motivation: Deep learning imputation methods model complex temporal dynamics using architectures including Transformers, VAEs, GANs, diffusion models, PFMs, and LLMs.They learn from observed values to estimate missing components in context.
- Survey scope and taxonomy: The survey proposes a taxonomy organized by imputation uncertainty and neural network architecture.It distinguishes predictive methods with fixed estimates from generative methods that provide distributions of possible values, and covers multiple neural architectures.
- Tools and resources: The survey reviews MTSI toolkits, emphasizing the PyPOTS Ecosystem’s integrated algorithms, standardized pipelines, and benchmarking resources.These components support accessible and reproducible MTSI research.
- Future directions: The survey identifies future directions involving missingness patterns, downstream task integration, and model scalability.The listed directions are presented as insights for advancing MTSI research.
2 Preliminary and Taxonomy
The paper formulates MTSI around incomplete observations, missingness mechanisms, and reconstruction objectives, then organizes deep-learning methods by uncertainty and neural-network architecture.
- Problem Definition: An incomplete time series is represented by observed values, an observation matrix, and a time-lag matrix encoding missingness and temporal gaps.The observation matrix marks each value as observed or missing, while the time-lag matrix captures elapsed time information.
- Problem Definition: MTSI constructs a parameterized imputation model to estimate missing values and form a reconstructed matrix from observed inputs.The imputed matrix combines observed entries with model-generated estimates through element-wise masking.
- Problem Definition: The imputation objective is either to approximate the complete data accurately or improve downstream-task performance relative to using the incomplete data.These objectives define reconstruction fidelity and task utility as two possible aims of imputation.
- Missing Mechanisms: Missing mechanisms describe the statistical relationship between observations and missingness, with MCAR and MAR treated as ignorable while MNAR is non-ignorable.Under MCAR and MAR, imputation can focus on the observed-data distribution; MNAR requires modeling the data and missingness jointly.
- Taxonomy: The proposed taxonomy classifies deep MTSI methods by imputation uncertainty and neural-network architecture, including predictive or generative and several architectural families.The architecture categories include RNN-, CNN-, GNN-, attention-, VAE-, GAN-, diffusion-, and large-model-based methods.
3 Predictive Methods
Predictive MTSI methods deterministically reconstruct missing values using reconstruction-based objectives, with models spanning recurrent, convolutional, graph, and attention-based architectures.
- Predictive Imputation: Predictive imputation methods produce deterministic estimates for identical missing components and therefore do not represent imputation uncertainty.Their typical training strategy minimizes reconstruction error using an absolute or squared error function.
- RNN-based Models: RNN-based methods model missing time-series values through recurrent processing, temporal decay, bidirectional inference, or cross-stream interpolation.Examples include GRU-D, Temporal belief memory, M-RNN, and BRITS.
- CNN-based Models: CNN-based methods support time-series imputation through local feature extraction, including architectures that transform one-dimensional series into two-dimensional representations.TimesNet uses the Fast Fourier Transform for this restructuring, while GP-VAE uses CNNs in its encoder and decoder.
- GNN-based Models: GNN-based methods treat time series as graph sequences and reconstruct missing values from learned node representations capturing temporal and spatial structure.GRIN uses a bidirectional graph recurrent network, while SPIN adds sparse spatiotemporal attention.
- Attention-based Models: Attention-based models generally outperform RNN- and CNN-based methods because attention handles long-range dependencies and enables parallel processing.DeepMVI combines transformer attention with convolutional techniques to model seasonal, local, and cross-dimensional information.
4 Generative Methods
Generative MTSI methods model distributions of observed data and generate varied estimates for missing observations, enabling uncertainty quantification. The survey covers VAE-, GAN-, and diffusion-based approaches, along with their trade-offs and design developments.
- Overview: Generative methods learn probability distributions from observed data and generate varied missing-value estimates, enabling imputation uncertainty quantification.The surveyed methods include VAE-, GAN-, and diffusion-based models.
- VAE-based Models: VAEs use encoder-decoder architectures and ELBO optimization to approximate data distributions through Gaussian latent spaces.GP-VAE adds a Gaussian-process prior to capture temporal dynamics, while evaluating the ELBO on observed features.
- GAN-based Models: GAN-based methods train generators against discriminators to produce synthetic data resembling real samples.GRUI-GAN combines adversarial distribution learning with input-noise optimization, while NAOMI uses multiresolution decoding to reduce error accumulation in long, highly missing sequences.
- Diffusion-based Models: Diffusion models progressively add and reverse noise to capture complex data distributions for imputation.CSDI conditions generation on observed data, SSSD replaces quadratic-complexity transformers with structured state space models, and other methods refine conditional or continuous-process modeling.
- Pros and Cons: VAE models can have limited generative capacity, GANs can be difficult to train, and diffusion models require substantial computation and may have boundary-coherence issues.These limitations affect the models’ ability to represent variability, maintain stability, or connect missing and observed regions.
5 Large Model-based Methods
Large model-based methods address complex temporal dependencies, diverse missingness patterns, and limited-domain generalization through pretrained foundation models and large language models. They provide richer temporal and cross-variable context, but require attention to computational resources and domain-specific constraints.
- Overview: Large model approaches target multiscale temporal dependencies, diverse missingness patterns, and robust generalization with limited domain data.The survey examines pretrained foundation models and large language models as complementary approaches to these challenges.
- Pre-trained Foundation Models: Pretrained foundation models use large-scale, diverse pretraining to improve generalization and adaptability across time-series domains.Timer learns temporal representations from many multivariate time series, while Timemixer++ mixes temporal and feature-wise representations with lower computational overhead.
- Large Language Models: Large language models exploit autoregressive sequence modeling and extensive parameters for multivariate time-series imputation.GPT4TS adapts GPT-2 by freezing its attention module, and retrieval augmentation supports irregular and sparse time-series data.
- Discussion: These approaches shift imputation from isolated missing points toward modeling rich temporal and cross-variable context.Their adoption still requires consideration of computational resources and domain-specific constraints, especially for complex missingness patterns.
6 Time Series Imputation Toolkits
MTSI toolkits range from basic statistical and machine-learning utilities to deep-learning libraries. PyPOTS provides a focused end-to-end toolbox, while the PyPOTS Ecosystem supports standardized benchmarking across many public datasets.
- Toolkit Landscape: Existing MTSI libraries provide naive processing, statistical methods, machine-learning algorithms, and deep-learning neural networks.The toolkit landscape spans multiple methodological levels rather than a single standardized implementation framework.
- Statistical Toolkits: imputeTS offers naive and common statistical algorithms but is limited to univariate time series.Its methods include mean values, last observation carried forward, linear interpolation, Kalman smoothing, and weighted moving averages.
- Statistical Toolkits: Impyute2, Autoimpute, and GluonTS provide simple or parametric imputation options for cross-sectional and time-series data.Examples include moving averages, polynomial and spline interpolation, dummy-value imputation, and causal mean imputation.
- PyPOTS Ecosystem: PyPOTS contains 37 imputation models for end-to-end modeling of partially observed time series.The associated TSI-Bench Suite provides standardized pipelines over 172 public time-series datasets for benchmarking imputation algorithms.
7 Future Direction
Future MTSI research must address missing-not-at-random mechanisms, downstream-task integration, and scalability. The survey highlights distribution shift under MNAR, open questions about impute-and-predict versus encode-and-predict, and the computational burden of deep models.
- Missingness Patterns: Most existing imputation algorithms assume MCAR or MAR, while real-world missingness can be MNAR and induce distributional shift.Under MNAR, missing high-value airflow observations can skew the observed distribution and introduce inductive bias in parameter estimation.
- Downstream Performance: The impute-and-predict and encode-and-predict paradigms remain alternatives, with the optimal approach for partially observed time series still open.End-to-end encoding is described as more promising when missingness patterns contain information useful for downstream tasks.
- Scalability: Deep imputation methods can be computationally less feasible for large-scale datasets than statistical and machine-learning counterparts.The survey points to parallel and distributed computing as needed for scalable deep imputation under rapidly increasing data volumes.
8 Conclusion
The survey organizes deep learning methods for multivariate time series imputation through a novel taxonomy and reviews their architectures, strengths, limitations, and applications. It identifies MNAR missingness, downstream-task integration, and scalability as key challenges, with pre-trained large models and multimodal learning as future directions.
- The survey proposes a novel taxonomy categorizing predictive and generative multivariate time series imputation methods, while also discussing large models.
- It provides a comprehensive overview of imputation architectures, including their strengths, limitations, and applications.
- Key challenges include handling MNAR missingness, integrating imputation with downstream tasks, and improving scalability.
- Future research should investigate large-scale pre-trained models and multimodal learning to improve robustness and real-world applicability.