Source-linked AI summary
MTSCI: A Conditional Diffusion Model for Multivariate Time Series Consistent Imputation
Jianping Zhou, Junhao Li, Guanjie Zheng, Xinbing Wang, Chenghu Zhou
TL;DR
Missing values compromise time-series analysis and downstream tasks, and existing imputation methods leave consistency between observed and imputed values and adjacent windows unaddressed. MTSCI uses complementary masking and adjacent-window conditioning in a conditional diffusion model; experiments report state-of-the-art results across multiple real-world datasets and missing scenarios.
Problem
Existing imputation methods leave consistency between observed and imputed values and between adjacent windows unaddressed.
Source: §1 · p. 2Method
MTSCI is a conditional diffusion model that uses complementary masked views with intra-contrastive loss and mixup-based conditioning from adjacent windows.
Source: §1 · p. 2, §4.1 · p. 3Results
Across three real-world datasets, MTSCI reports average improvements of 17.88% in MAE, 15.09% in RMSE, and 13.64% in MAPE over baseline methods.
Source: §1 · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
Missing values are prevalent in multivariate time series, compromising the integrity of analyses and degrading the performance of downstream tasks. Consequently, research has focused on multivariate time series imputation, aiming to accurately impute the missing values based on available observations. A key research question is how to ensure imputation consistency, i.e., intra-consistency between observed and imputed values, and inter-consistency between adjacent windows after imputation. However, previous methods rely solely on the inductive bias of the imputation targets to guide the learning process, ignoring imputation consistency and ultimately resulting in poor performance. Diffusion models, known for their powerful generative abilities, prefer to generate consistent results based on available observations. Therefore, we propose a conditional diffusion model for Multivariate Time Series Consistent Imputation (MTSCI). Specifically, MTSCI employs a contrastive complementary mask to generate dual views during the forward noising process. Then, the intra contrastive loss is calculated to ensure intra-consistency between the imputed and observed values. Meanwhile, MTSCI utilizes a mixup mechanism to incorporate conditional information from adjacent windows during the denoising process, facilitating the inter-consistency between imputed samples. Extensive experiments on multiple real-world datasets demonstrate that our method achieves the state-of-the-art performance on multivariate time series imputation task under different missing scenarios. Code is available at https://github.com/JeremyChou28/MTSCI.
1 INTRODUCTION
Missing values impair downstream time-series tasks, while existing imputation methods do not address consistency between observed and imputed values or adjacent windows. MTSCI targets both forms of consistency with a conditional diffusion model and reports state-of-the-art imputation performance.
1 INTRODUCTION
- Missing values impair downstream task performance and make time-series integrity analyses inapplicable, motivating imputation from available observations.Source: §1 · p. 1
- Earlier methods use simulated imputation targets to guide learning but neglect consistency between observed and imputed values and between adjacent windows.The paper distinguishes intra-consistency within a sequence from inter-consistency across adjacent windows.Source: §1 · pp. 1–2
- Across three real-world datasets, MTSCI reports average improvements of 17.88% in MAE, 15.09% in RMSE, and 13.64% in MAPE over baseline methods.The introduction presents these as experimental results for the proposed method.Source: §1 · p. 2
- MTSCI uses complementary masked views and an intra-contrastive loss for intra-consistency, while adjacent-window conditional information supports inter-consistency through mixup.The complementary-mask strategy is used during noising; the adjacent-window information is incorporated during denoising.Source: §1 · p. 2, Figure 1 · p. 2
- The paper presents its contributions as defining intra- and inter-consistency, proposing MTSCI, and evaluating it across real-world datasets and missing-data scenarios.Source: §1 · p. 2
2 RELATED WORK
Prior multivariate time-series imputation spans statistical, machine-learning, deterministic deep-learning, and probabilistic generative approaches. The paper identifies consistency between observed and imputed values and across adjacent sequences as an unmet imputation concern.
2 RELATED WORK
- Statistical, machine-learning, deterministic deep, and probabilistic generative methods form the four categories of prior multivariate time-series imputation.The cited examples include mean, KNN, MICE, BRITS, SAITS, CSDI, and PriSTI.Source: §2 · p. 2
- Statistical and classical machine-learning methods rely on statistical indicators or linear-dynamics assumptions, while deterministic deep models estimate points and can miss imputation uncertainty.Source: §2 · p. 2
- Prior consistency strategies in time-series representation and contrastive learning focus on complete series, while imputation methods do not address intra- and inter-consistency.Representation work examines adjacent-window patterns; other methods use augmented views or time-frequency transformations.Source: §2 · pp. 2–3
3 PRELIMINARIES
The preliminaries define multivariate time-series imputation and outline diffusion and conditional-diffusion processes. Together, they frame the task as estimating missing values from observed values using a probabilistic model.
3.1 Problem Definition
- The imputation task estimates missing values or their distributions from a multivariate series and its observed/missing mask, by approximating a conditional distribution or minimizing error at missing positions.The mask marks observed entries with 1 and missing entries with 0.Source: §3.1 · p. 3
3.2 Diffusion Models
- A diffusion model gradually adds Gaussian noise in a forward process and uses a learned denoising network in the backward process.The preliminaries describe both noise-prediction and data-estimation parameterizations.Source: §3.2 · p. 3
3.3 Conditional Diffusion Models
- Conditional diffusion imputation uses observed values as conditioning information while noising and denoising the missing values.Repeated denoising steps produce the imputed values.Source: §3.3 · p. 3
4 METHODOLOGY
The methodology section outlines MTSCI’s model overview, consistency mechanisms in noising and denoising, and training and inference procedures.
4 METHODOLOGY
- The section first presents an overview of MTSCI for multivariate time-series imputation.Source: §4 · p. 3
- It then describes contrastive consistency in the forward noising process and consistency-assured denoising.Source: §4 · p. 3
- The section concludes by outlining the algorithm procedures for training and inference.Source: §4 · p. 3
4.1 Overview
MTSCI is a conditional diffusion model for multivariate time-series imputation that addresses consistency within a window and between adjacent windows. It uses complementary-view contrastive learning and context-window conditioning, and combines denoising and intra-contrastive losses.
4.1 Overview
- MTSCI uses a conditional diffusion model to impute a sampled window while targeting consistency within that window and across adjacent windows.The intra-contrastive module compares two views; the inter-consistency condition network incorporates context-window information.Source: §4.1 · p. 3
- The intra-contrastive module promotes consistency between imputed and observed values by calculating contrastive loss between two views.The model’s objective combines denoising loss L𝜃 and intra-contrastive loss L𝐶𝐿.Source: §4.1 · p. 3, Figure 2 · p. 4
- The inter-consistency condition network incorporates context-window information to support consistency between adjacent windows.Source: §4.1 · p. 3, Figure 2 · p. 4
4.2 Contrastive Consistency on Noising Process
MTSCI uses complementary masks to create two views of a sampled window, aiming to make imputed values consistent with observed values. The views are noised for the subsequent process.
4.2 Contrastive Consistency on Noising Process
- The complementary-mask strategy generates paired samples with complementary imputation targets and observations, so imputed values can be trained to agree with observed values.The paper motivates this strategy as addressing inconsistency that may otherwise lead to overfitting outliers and deviations from ground truth.Source: §4.2 · p. 4
4.2.1 Contrastive Complementary Mask.
- A random mask m splits the sampled window into two views, m⊙x and (1−m)⊙x.The sampled window is x_{t−L:t}; the adjacent context window is x_{t:t+L}.Source: §4.2.1 · p. 4
4.2.2 Noising.
- Each view’s imputation target is noised using ε sampled from N(0, I) with the same size as the target.The passage identifies a separate conditional observation for the example view and refers to the noising definition in equation (3).Source: §4.2.2 · p. 4
4.3 Consistency-Assured Denoising Process
The denoising process combines an intra-view contrastive loss with context-window conditioning. These mechanisms target consistency between imputed and observed values and between adjacent windows, while a denoising network predicts noise at missing positions.
4.3.1 Intra Contrastive Module(Intra-consistency).
- The intra-contrastive loss compares embeddings from complementary views, constraining their representation similarity so imputed values can reconstruct observed values.A batch of N samples yields 2N views after complementary masking.Source: §4.3.1 · p. 4
4.3.2 Inter-consistency Condition Network(Inter-consistency).
- The inter-consistency network mixes sampled-window conditional observations with context-window information to condition imputation across adjacent windows.The context window is available during training but not inference; at inference, all elements of the mixing matrix are set to 1.Source: §4.3.2 · pp. 4–5
- The mixed conditional representation is used by the denoising-network encoder to capture dependencies between observed and missing values.Source: §4.3.2 · p. 5
4.3.3 Denoising Network.
- The denoising network takes the noised sampled window and conditional representation to predict noise at missing positions and generate imputed values.Its encoder uses transformer and inverted-transformer blocks to capture temporal and variable dependencies; the decoder produces predicted noise through a feed-forward network.Source: §4.3.3 · p. 5
4.4 Training
MTSCI training combines denoising and intra-contrastive losses. Algorithm 1 takes incomplete time series and a mask as input and produces an optimized denoising network.
4.4 Training
- The total training loss adds denoising loss L𝜖 and intra-contrastive loss L𝐶𝐿 using a weighted coefficient λ.Source: §4.4 · p. 5
- Algorithm 1 takes incomplete time series, a mask matrix, diffusion-step count K, and noise levels ᾱ_k as input, and outputs an optimized denoising network.Source: §4.4 · p. 5
- The training procedure generates two complementary-mask views, obtains a diffusion-step embedding, passes it to the encoder, and calculates the total loss.The algorithm’s listed steps include calculating the total loss using equation (22).Source: §4.4 · p. 5
4.5 Inference
MTSCI inference sets every mixing coefficient to 1 and repeatedly denoises until step k equals 1 to obtain the final predicted noise.
4.5 Inference
- At inference, all mixing coefficients are set to 1, and denoising repeats until k equals 1 to produce the final predicted noise.The inference procedure is described in Algorithm 2.Source: §4.5 · p. 5
5 EXPERIMENTS
Across multiple datasets and missing patterns, MTSCI achieves state-of-the-art imputation performance. The experiments evaluate its consistency components, case-study behavior, generalization, and sensitivity to hyperparameters.
5.1 Experimental Setup
- Experiments use ETT, Weather, and METR-LA, 13 baselines, MAE/RMSE/MAPE, nonoverlapping 24-step windows, and point- and block-missing evaluation.The study examines overall performance, consistency, missing scenarios, and hyperparameters.Source: §5 · pp. 5–6, §5.1 · p. 6
5.2 Overall Performance(RQ1)
- MTSCI achieves state-of-the-art performance across multiple datasets for point and block missing, averaging improvements of 17.88% in MAE, 15.09% in RMSE, and 13.64% in MAPE.The overall-performance tables cover three datasets and report results averaged over five runs.Source: §5.2 · p. 6, Table 2 · p. 7, Table 3 · p. 7
- MTSCI improves over deterministic methods by 42.07% in MAE, 24.15% in RMSE, and 39.76% in MAPE on average, and consistently outperforms deep generative methods on several datasets.The authors attribute the generative-model comparison to complementary-mask views and adjacent-window mixup conditioning.Source: §5.2 · pp. 6–7
5.3 Ablation Study(RQ2)
- Removing complementary-mask contrastive views or adjacent-window conditioning deteriorates performance, while using both modules jointly improves imputation.The authors connect the views to mutual reconstruction of observed and imputed values, and adjacent windows to contextual consistency constraints.Source: §5.3 · pp. 7–8
- Predicting noise yields better performance than predicting missing values with the same denoising architecture.The authors suggest Gaussian noise supports complementary views in predicting and reconstructing each other; observed and missing values may have different distributions.Source: §5.3 · p. 8, Table 4 · p. 8
5.4 Case Study(RQ2)
- On three block-missing snapshots, MTSCI visually outperforms CSDI and its variants, and it also outperforms CSDI on consistency measured by CRPS.The visualization highlights consistent trends between imputed and observed values; lower CRPS indicates better consistency.Source: §5.4 · p. 8, Table 5 · p. 8, Figure 4 · p. 9
5.5 Sensitivity Analysis(RQ3)
- MTSCI outperforms CSBI and CSDI across tested missing ratios and mask patterns on Weather, and performs relatively well when training and testing patterns differ.The authors note that performance does not consistently decline as missing ratios rise and that the consistency strategy helps alleviate this issue to some extent.Source: §5.5 · p. 8, Table 6 · p. 8, Table 7 · p. 8
5.6 Hyperparameter Study(RQ4)
- Hidden dimension size, encoder depth, maximum noise level, and contrastive-loss weight each affect performance and require tuning.Larger encoder depth can improve dependency capture but increases parameters and computation; moderate noise and small-scale adjustment of the loss weight are favored.Source: §5.6 · p. 9, Figure 5 · p. 9
6 CONCLUSION
MTSCI combines intra- and inter-consistency mechanisms in a conditional diffusion model for multivariate time-series imputation. Experiments show state-of-the-art performance on multiple real-world datasets under varied settings, and future work will extend the method to more complex missing-data scenarios.
6 CONCLUSION
- MTSCI achieves state-of-the-art performance on multiple real-world datasets under various experimental settings.The paper plans to extend the method to more complex missing-data scenarios.Source: §6 · p. 9
- A complementary-mask strategy introduces intra contrastive loss to ensure mutual consistency between imputed and observed values.Source: §6 · p. 9
- An inter-consistency condition network uses mixup to incorporate information from adjacent windows and facilitate imputation.Source: §6 · p. 9
- The paper frames intra- and inter-consistency as approaches for improving imputation performance.Source: §6 · p. 9