PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Imputation of Missing Values in Time Series Health Data: A Review with Benchmarking

Maksims Kazijevs, Manar D. Samad

arXiv:2302.10902v2cs.LGcs.AI

TL;DR

Missing-value imputation in multivariate time series matters for data quality and reliable predictive models, yet deep methods have been evaluated on few datasets and limited missingness settings. This survey benchmarks methods in six data-centric experiments across five health datasets and finds no single method best across all five. Performance varies with data and missingness characteristics, supporting data-centric method selection.

  • Problem

    Deep imputation methods have limited evaluation across datasets, missingness rates, and missingness types, although imputation affects data quality and predictive reliability.

    Source: §1 · p. 1

  • Method

    The survey conducts six data-centric experiments benchmarking state-of-the-art deep imputation methods on five time-series health datasets.

    Source: §6 · p. 22

  • Results

    No single imputation method is best across all five datasets, and performance varies with data types, variable statistics, missing rates, and missingness types.

    Source: §5 · p. 21

  • Takeaways & Limitations

    Method selection should account for data-centric factors to support data quality and reliable data-driven outcomes.

    Source: §6 · p. 22

    Deep learning methods are more time-consuming than MICE and other conventional methods.

    Source: §5.1 · p. 22

Abstract

from arXiv · show

The imputation of missing values in multivariate time series (MTS) data is critical in ensuring data quality and producing reliable data-driven predictive models. Apart from many statistical approaches, a few recent studies have proposed state-of-the-art deep learning methods to impute missing values in MTS data. However, the evaluation of these deep methods is limited to one or two data sets, low missing rates, and completely random missing value types. This survey performs six data-centric experiments to benchmark state-of-the-art deep imputation methods on five time series health data sets. Our extensive analysis reveals that no single imputation method outperforms the others on all five data sets. The imputation performance depends on data types, individual variable statistics, missing value rates, and types. Deep learning methods that jointly perform cross-sectional (across variables) and longitudinal (across time) imputations of missing values in time series data yield statistically better data quality than traditional imputation methods. Although computationally expensive, deep learning methods are practical given the current availability of high-performance computing resources, especially when data quality and sample size are highly important in healthcare informatics. Our findings highlight the importance of data-centric selection of imputation methods to optimize data-driven predictive models.

1 Introduction

Missing values can undermine predictive outcomes, and imputation choices affect the quality of model inputs. The introduction motivates time-series methods that account for dependencies across both variables and time, while noting gaps in existing benchmarking.

1 Introduction

  • Imputation method selection regulates input data quality and ultimately affects the accuracy and robustness of data-driven predictive models.Missing values are described as a major obstacle to reliable predictive outcomes.Source: §1 · p. 1
  • Time-series health data contain missingness across variables and time, so imputation must capture both variable and temporal dependencies.The added time dimension creates computational challenges beyond multivariate tabular imputation.Source: §1 · p. 1
  • Prior surveys and benchmarks leave limited evidence on deep methods across multiple health datasets and varied missingness scenarios.The introduction describes this paper as among the first to comprehensively review and benchmark state-of-the-art deep methods on multiple time-series health datasets.Source: §1 · p. 2

2 Background review

The background contrasts cross-sectional and time-dependent imputation, then reviews missingness types and statistical, machine-learning, and deep approaches. It emphasizes that methods need to address both temporal and variable dependencies and be tested under realistic missingness conditions.

2.1 Cross-sectional imputation

  • Cross-sectional imputation models relationships among variables, commonly using MICE, while deep approaches include autoencoders, generative models, CNNs, and RNNs.MICE estimates a variable with missing values from other observed or imputed variables.Source: §2.1 · pp. 2–3

2.1.1 Missing value types

  • MCAR, MAR, and MNAR distinguish missingness independent of data, dependent on observed data, or dependent on missing values themselves.Time-series methods are rarely evaluated across these types, although medical missingness is not always completely random.Source: §2.1.1 · p. 3

2.2 Time series data imputation

  • Longitudinal imputation models each variable as a function of time, using observed values to estimate missing values at other time points.The review describes this as a complement to cross-sectional imputation for time-series data.Source: §2 · p. 2, §2.2 · p. 3

2.2.1 Statistical and machine learning based methods

  • Statistical and machine-learning approaches combine temporal and cross-sectional information, including 3D-MICE, MICE-DA, MD-MTS, SMILES, and TA-DualCV.3D-MICE combines flattened-data MICE and Gaussian-process longitudinal estimates; TA-DualCV fuses cross-visit modules with patient-specific Gaussian-process estimates.Source: §2.2.1 · p. 4

2.2.2 The DACMI challenge for missing value imputation

  • DACMI challenge methods augment cross-sectional features with temporal features, but their reported benchmarking uses only the DACMI EHR dataset.MD-MTS uses augmented features with LightGBM; MICE-DA combines local temporal and global patient-similarity features, while SMILES uses window-based features and XGBoost.Source: §2.2.2 · p. 4

2.2.3 Deep time series imputation methods

  • Deep time-series methods use architectures such as bidirectional RNNs and LSTMs to model temporal and cross-sectional dependencies, with some also incorporating time gaps or patient context.BRITS models uneven time gaps; CATSI adds a context vector and uses a learned trade-off to fuse longitudinal and cross-sectional estimates, while TAME uses time-aware attention and multimodal patient information.Source: §2.2.3 · pp. 4–6

2 Background review

  • Time-series health data add patient, time, and variable dimensions, with differing follow-up counts and measurements that may be absent at some time points.The review frames time-series imputation as needing both cross-sectional and longitudinal information.Source: §2 · p. 2, Figure 2 · p. 6

2.3 Health records and biomedical sensor data

Health records and biomedical sensor data provide longitudinal measurements but are often incomplete and irregular. The section discusses non-random missingness, risks of excluding incomplete cases, limits in prior benchmarking, and the challenge of high missing rates.

2.3 Health records and biomedical sensor data

  • EHRs store diagnostic, sensor, and imaging measurements for patient care, but their complex and incomplete structure requires careful curation for predictive modeling.The section also identifies biomedical sensor data such as ECG and EEG as widely used in medical practice.Source: §2.3 · p. 6

2.3.1 Irregular and non-random missing values

  • EHR visits and measurements occur at irregular intervals, and missingness can arise from clinical decisions, recording failures, or sensor and transmission problems.The passage states that missingness patterns are therefore never entirely random, while existing time-series methods assume MCAR.Source: §2.3.1 · p. 7

2.3.2 Predictive modeling of data with missing values

  • Complete-case exclusions can retain a selective patient population and create patient-selection bias, while reducing data available to data-hungry deep models.The section notes that imputation choices are often arbitrary and unvalidated.Source: §2.3.2 · p. 7
  • Prior methods lack rigorous benchmarking across multiple datasets and varied scenarios, including missingness rates and types.Table 1 summarizes methods by missing percentages, dataset counts, domain, irregularity, and downstream classification validation.Source: §2.3.2 · pp. 7–8, Table 1 · p. 7

2.3.3 Missing rates in health data

  • Health variables can exceed 50% missingness, and the effect of varying missing rates on time-series imputation methods is described as unknown.The section argues that accurate imputation at high missing rates matters for retaining sample size and potentially important predictor variables.Source: §2.3.3 · p. 8

3 Methodology

The methodology section introduces the datasets, selected imputation methods, and rationale for six evaluation experiments.

3 Methodology

  • The study evaluates time-series imputation methods using selected datasets and methods across six experiments.The section is framed around the dataset choices, method selection, and justification for those experiments.Source: §3 · p. 8

3.1 Time series health data sets

The benchmark uses five health datasets spanning electronic health records, synthetic ICU data, physiological signals, and heart sounds. Their sample sizes, variables, and time resolutions differ substantially.

3.1 Time series health data sets

  • DACMI contains 13 laboratory variables from 8,267 ICU patients, with ground-truth indices; records lacking native ground truth are excluded.The dataset is derived from MIMIC-III and excludes patients with fewer than ten time points or follow-up visits.Source: §3.1 · p. 8
  • The synthetic hypotension dataset has 3,910 patients with nine numeric variables at hourly intervals over 48 hours; sepsis has 2,164 patients and 30 selected numeric variables across 20 four-hour steps.Both datasets are generated from MIMIC-III data, and the synthetic datasets undergo three-stage validation.Source: §3.1 · pp. 8–9
  • IEEEPPG provides 3,096 signal samples with five variables and 1,000 time steps, but this study uses only the first 50 time points.The modified dataset contains two PPG signals and three-axis acceleration signals, sampled from running subjects.Source: §3.1 · p. 9
  • The heartbeat dataset contains 409 heart-sound samples represented by 61 spectrogram frequency bands, truncated to the first 50 of 405 time steps.Recordings are five seconds long and include healthy subjects and patients with confirmed cardiac diagnoses.Source: §3.1 · p. 9

3.2 Time series imputation methods

The benchmark selects recent deep-learning imputation methods with available code, alongside two baselines. It includes methods with different approaches to combining temporal and cross-variable information.

3.2 Time series imputation methods

  • The benchmark restricts its state-of-the-art selection to deep-learning methods published in the last five years and excludes methods without available source code.The selection rules also exclude non-deep methods and older methods such as GRU-D and M-RNN.Source: §3.2 · p. 9
  • MICE and basic LSTM-based imputation serve as the two baselines.MICE is included as a popular health-science imputation method, while LSTM-based imputation represents a basic deep method for time series.Source: §3.2 · p. 9
  • The selected methods include BRITS, CATSI variants, NAOMI, and TAME, with CATSI split into temporal and cross-sectional components for comparison.The benchmark also compares bidirectional CATSI-LSTM with conventional unidirectional LSTM; DETROIT and GP-VAE are assessed in downstream classification tasks.Source: §3.2 · pp. 9–10

3.3 Experiments

The experiments benchmark imputation across five health datasets and examine initialization, sequence length, imputation direction, missingness rate and type, and downstream classification. The study frames these tests around gaps in the robustness and generalizability of existing evaluations.

3.3 Experiments

  • The experiments address whether method performance generalizes across datasets, sequence lengths, missingness rates and types, variables, and downstream prediction.The motivating gaps include limited testing at low rates with randomly introduced missingness and uncertainty about whether imputation accuracy translates to classification performance.Source: §3.3 · p. 10

3.3.1 Effects of initializing missing values

  • Four initialization approaches are compared: mean, median, piece-wise time interpolation, and delta initialization based on the time gap.The delta approach is described as using the mean for longer gaps and the next time point for shorter gaps; piece-wise interpolation estimates a value between adjacent timestamps.Source: §3.3.1 · p. 10

3.3.2 Effects of time series length

  • The time-length experiment truncates series to 5, 10, 15, 20, or 25 time steps and zero-pads shorter series.It tests whether methods remain stable with shorter or longer sequences, given the potential difficulty of modeling long sequences and limited temporal information in short ones.Source: §3.3.2 · p. 10

3.3.3 Cross-sectional versus longitudinal versus hybrid imputation methods

  • The cross-sectional, longitudinal, and hybrid comparison uses piece-wise interpolation as a longitudinal baseline and two cross-sectional chained-equation methods built on gradient-boosting regressors.CE-GBT-I trains one regressor per variable and iteratively updates test imputations; CE-GBT-II also iteratively replaces missing values during repeated training.Source: §3.3.3 · pp. 10–11

3.3.4 Effects of percentage of missing values

  • The missing-rate experiment evaluates imputation accuracy at rates up to 80%, beyond the 5%–10% rates commonly assumed in prior evaluations.The study motivates testing whether methods retain performance at higher missingness rates.Source: §3.3.4 · p. 11

3.3.5 Effects of missing value types

  • The missing-type experiment simulates MCAR by random cell removal, MNAR by removing target-variable tail values, and MAR by iteratively removing target values associated with extremes in three observed variables.Each simulated type is tested at a specified total missing rate.Source: §3.3.5 · p. 12

3.3.6 Classification of imputed data

  • The classification experiment compares data completed by deep imputation methods with complete ground-truth data and unimputed data with missing values.It tests whether imputation accuracy corresponds to downstream classification performance.Source: §3.3.6 · p. 12

3.4 Model evaluation

Evaluation uses shared train-test folds and normalized error to compare imputation methods, with weighted F1 for classification. Method ranks are aggregated across variables, and selected experiments use Wilcoxon signed-rank tests.

3.4 Model evaluation

  • All datasets use an 80%–20% train-test split and, except in missing-rate and missing-type experiments, 5% MCAR; imputation accuracy is evaluated against test-set ground truth.The same folds are used across experiments, and NRMSD normalizes errors across variable scales while masking non-missing entries.Source: §3.4 · p. 12
  • Methods are ranked by NRMSD for each variable, and their average rank is computed across variables; classification performance is reported with weighted F1.The initialization and classification experiments use Wilcoxon signed-rank tests for statistical comparisons.Source: §3.4 · pp. 12–13
  • Table 3 summarizes initialization methods by average NRMSD and average rank across imputation models and variables; Table 4 reports the best initialization for each model-dataset pair.Table 3 averages over seven imputation models, while Table 4 averages the selected initialization’s rank across variables within a dataset.Source: Table 3 · p. 13, Table 4 · p. 13

4 Experimental results

Results vary across datasets, variables, series lengths, and missingness regimes; no single imputation method is universally best. The preferred method also varies in downstream classification comparisons.

4.1 Effects of initializing missing values

  • Median ranks best on DACMI (2.1); mean performs best on sepsis (NRMSD 0.233 (0.03), rank 1.6) and hypotension (0.211 (0.06), rank 1.5); IEEEPPG piece-wise significantly beats mean, median, and delta (p<0.05).Separately, delta initialization is significantly better than median initialization.Source: §4.1 · pp. 13–14

4.2 Effects of time series length

  • CATSI is generally best beyond 15 time points; short-series winners vary, including CATSI-MLP on DACMI and hypotension at five points.The experiment used 5% MCAR missingness and compared series lengths.Source: Table 5 · p. 14, §4.2 · p. 15

4.3.1 DACMI data set

  • DACMI favors hybrid methods for 8 of 13 variables, while sepsis favors them for 17 of 30; cross-sectional methods lead for 12 sepsis variables.On DACMI, two highly correlated variables (HCT and HGB; correlation 0.96) favor cross-sectional CATSI-MLP.Source: §4.3.1 · pp. 15–16, §4.3.2 · p. 16

4.3.3 Hypotension data set

  • Hybrid methods lead all but MAP in hypotension; piece-wise interpolation leads every IEEEPPG variable, whose highest pairwise correlation is 0.31.The hypotension passage reports a 0.80 correlation between MAP and DBP.Source: Table 7 · p. 16, §4.3.3 · p. 16, Table 8 · p. 16

4.4 Effects of missing values rates and types

  • On DACMI, BRITS overtakes CATSI above 50% MAR/MCAR missingness and leads MNAR; BRITS also generally leads sepsis, while hypotension winners vary by rate and type.For hypotension, TAME leads at 30%–70% missingness for MAR and MCAR, while BRITS generally leads for MNAR.Source: Figure 3 · p. 17, §4.4 · pp. 17–18, Figure 4 · p. 18

4.3.5 Heartbeat data set

  • No single method is superior for imputing every heartbeat variable; CATSI leads 24 variables, while piece-wise interpolation leads 17.Variable-specific selection among cross-sectional, longitudinal, and hybrid methods can optimize overall accuracy.Source: §4.3.5 · p. 17

4.4 Effects of missing values rates and types

  • Heartbeat favors CATSI across MAR and MCAR rates; IEEEPPG winners vary by missingness type and rate, shifting among CATSI-LSTM, TAME, and BRITS.For IEEEPPG, BRITS overtakes TAME at missing rates of 50% or higher for MNAR.Source: §4.4 · p. 18, Figure 4 · p. 18, Figure 5 · p. 19

4.5.1 Classification accuracy

  • Weighted F1 was 0.724 (0.004) for sepsis CATSI-LSTM and LSTM, 0.811 (0.002) for hypotension TAME, and 0.659 (0.061) for heartbeat CATSI-LSTM.The reported comparison used 20% MCAR missingness and mean initialization.Source: Table 9 · p. 19, §4.5.1 · pp. 19–20

5 Discussion

The discussion finds that imputation choices should reflect data characteristics: no method is best across all health datasets, missingness conditions, and variables. It recommends hybrid methods for many healthcare settings while noting computational and implementation trade-offs.

5 Discussion

  • No single method is best across health datasets; imputation performance varies with data type, variable statistics, missingness rate, and missingness type.The discussion favors hybrid methods for EHR data and longitudinal approaches for time-series signals.Source: §5 · p. 21
  • MNAR imputation error is substantially higher than for MAR or MCAR, and MCAR benchmarks may not reflect real-world EHR performance.The discussion notes that missing values in EHR data do not appear randomly as MCAR.Source: §5 · p. 21
  • BRITS generally performs best for MNAR across missing rates; at rates above 50%, BRITS performs well while TAME improves and CATSI performs worse.CATSI generally performs much better below 50% than at higher rates.Source: §5 · p. 21
  • Imputation’s effect on downstream classification is mixed: EHR results favor several deep methods, while imputed sensor data can outperform complete signals.The authors suggest noise filtering as a possible explanation for the sensor-data pattern.Source: §5 · p. 21

5.1 Recommendations for health scientists

  • Deep methods take longer than conventional baselines, but the authors recommend high-performance computing and prioritize data quality for this one-time preparation step.They also identify transfer of open-source tools to researchers trained in R and statistical software as an open problem.Source: §5.1 · p. 22
  • For practice, the authors recommend interpolation followed by MICE as a basic strategy, or CATSI/BRITS when hybrid imputation is suitable.TAME and DETROIT require ground-truth training values, which may be unavailable in practice.Source: §5.1 · p. 22

6 Conclusions

The paper benchmarks deep time-series imputation methods across six data-centric experiments and concludes that method selection should account for dataset characteristics. It identifies interactions among experimental conditions as a future research direction.

6 Conclusions

  • Six experiments compare imputation methods across data types, variable statistics, time-series length, missingness rates and types, and downstream classification.The study presents one of the first surveys to benchmark state-of-the-art deep methods for time-series imputation.Source: §6 · p. 22
  • The findings support data-centric method selection rather than arbitrary imputation to ensure data quality and reliable data-driven outcomes.Source: §6 · p. 22
  • Future work may examine interactions among the six experimental conditions to optimize imputation-method selection.Source: §6 · p. 22

Appendix A. Effects of missing value initialization methods

The appendix table describes how missing-value initialization methods are evaluated across models and datasets.

Appendix A. Effects of missing value initialization methods

  • Table A.1 reports average NRMSD and average rank across variables for initialization methods, imputation models, and datasets.Source: Table A.1 · p. 28

Appendix B. Correlation between variables versus imputation accuracy (NRMSD scores) to demonstrate the selection of cross-sectional (MLP) and longitudinal (CATSI-LSTM) imputation methods for individual variables.

The appendix figure relates variable correlations to individual-variable imputation error in two datasets.

Appendix B. Correlation between variables versus imputation accuracy (NRMSD scores) to demonstrate the selection of cross-sectional (MLP) and longitudinal (CATSI-LSTM) imputation methods for individual variables.

  • Figure B.1 plots HCT correlations against variable NRMSD scores in DACMI and shows the corresponding DBP plot for Hypotension.Source: Figure B.1 · p. 29

Related papers

Loading 2302.10902v2…