PaperlayerView exact revision on arXiv

Source-linked AI summary

BRITS: Bidirectional Recurrent Imputation for Time Series

Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, Yitan Li

arXiv:1805.10572v1cs.LGstat.ML

TL;DR

Missing values are common in time-series classification and regression, while existing imputers often impose strong assumptions about the data-generating process. BRITS learns missing values in a bidirectional recurrent system and jointly performs imputation with prediction, outperforming state-of-the-art methods across three real-world datasets.

  • Problem

    Time-series applications need to predict labels from multivariate sequences while imputing missing values, but many existing methods impose strong assumptions on the data-generating process.

  • Method

    BRITS treats missing values as variables in a bidirectional RNN graph, learns them through forward and backward recurrent dynamics, and jointly performs imputation with classification or regression.

  • Results

    BRITS outperforms state-of-the-art models for imputation and classification/regression accuracies across air-quality, health-care, and human-activity localization datasets.

  • Takeaways & Limitations

    The method provides a data-driven imputation procedure for multiple correlated time series without a specific assumption over the data.

  • Takeaways & Limitations

    The paper states that the attention-mechanism design is outside its scope.

Abstract

from arXiv · show

Time series are widely used as signals in many classification/regression tasks. It is ubiquitous that time series contains many missing values. Given multiple correlated time series data, how to fill in missing values and to predict their class labels? Existing imputation methods often impose strong assumptions of the underlying data generating process, such as linear dynamics in the state space. In this paper, we propose BRITS, a novel method based on recurrent neural networks for missing value imputation in time series data. Our proposed method directly learns the missing values in a bidirectional recurrent dynamical system, without any specific assumption. The imputed values are treated as variables of RNN graph and can be effectively updated during the backpropagation.BRITS has three advantages: (a) it can handle multiple correlated missing values in time series; (b) it generalizes to time series with nonlinear dynamics underlying; (c) it provides a data-driven imputation procedure and applies to general settings with missing data.We evaluate our model on three real-world datasets, including an air quality dataset, a health-care data, and a localization data for human activity. Experiments show that our model outperforms the state-of-the-art methods in both imputation and classification/regression accuracies.

1 Introduction

BRITS addresses missing values in correlated time series with a bidirectional RNN that learns imputations without specific data assumptions and jointly supports downstream prediction. It reports stronger imputation and classification/regression accuracy than state-of-the-art methods across three real-world datasets.

  • Motivation: Missing time-series values are common and can significantly harm downstream classification and regression.Prior methods often rely on strong assumptions, including linear dynamics or smoothness during missing periods.
  • Method: BRITS uses a bidirectional RNN to learn missing values directly in recurrent dynamics without specific assumptions about the data.The approach is intended to handle nonlinear dynamics and general missing-data settings.
  • Method: Missing values are treated as variables in the bidirectional RNN graph and updated through delayed forward and backward gradients with consistency constraints.The paper states that this makes missing-value estimation more accurate.
  • Joint learning: BRITS jointly performs imputation and classification/regression in one neural graph.The paper states that joint supervision alleviates error propagation and improves missing-value estimation.
  • Evaluation: BRITS was evaluated on air-quality, health-care, and human-activity localization datasets, outperforming state-of-the-art models for imputation and classification/regression accuracies.The reported evaluation covers three real-world datasets and both imputation and downstream prediction tasks.

2 Related Work

Related work includes statistical, interpolation, autoregressive, state-space, chained-equation, graphical, and data-driven approaches to time-series imputation. Recurrent neural-network methods jointly trained with prediction components form a more recent line of work, including GRU-D.

  • Classical and data-driven methods: Existing imputation research spans interpolation, autoregressive and state-space models, MICE, graphical models, and data-driven methods.These approaches differ in whether they exploit local smoothness, parameterized dynamics, chained equations, latent variables, or external and cross-variable information.
  • Classical methods: Interpolation methods reconstruct missing values with local smooth curves but discard relationships between variables over time.Autoregressive and state-space methods instead fit parameterized temporal models after addressing non-stationarity.
  • Recurrent methods: Recent recurrent methods train imputation components together with classification or regression components, which the paper says significantly boosts accuracy.GRU-D imputes health-care data smoothly by combining a variable's last observed value with the global mean.

3 Preliminary

The paper formulates multivariate time-series prediction with missing values as a joint task: predict a scalar or vector label while imputing the missing entries in the input sequence. It represents observations, timestamps, masks, and elapsed gaps explicitly.

  • Definitions: A multivariate time series is a sequence of T observations, each containing D features and occurring at a timestamp that may have irregular gaps.The paper denotes the sequence as X = {x1, x2, . . . , xT}.
  • Missingness: Unexpected events such as equipment damage or communication errors can make features within an observation missing.The paper illustrates this with a missing feature in one observation.
  • Missingness representation: A masking vector mt represents whether each feature at time t is observed or missing.The supplied definition states that the mask distinguishes unobserved features from observed ones.
  • Time gaps: The time gap δ^d_t records the elapsed time from feature d's last observation to the current timestamp st.For the second feature at x6, the example gives δ^2_6 = 13 because the last observation occurred at s2 = 2 and s6 = 15.
  • Problem formulation: The learning objective is to predict a scalar or vector label y from X while imputing X's missing values as accurately as possible.The paper describes this as a joint multi-task learning problem for classification or regression and imputation.

4 BRITS

BRITS imputes missing values in correlated time series through bidirectional recurrent dynamics, treating estimates as trainable variables and combining temporal and feature-based information. Its design addresses delayed errors, correlated features, and irregular sampling.

  • Unidirectional imputation: RITS-I uses predecessor-based recurrent dynamics and validates missing estimates through later observations, but delayed errors can slow convergence and amplify early mistakes.For a missing run x5–x7, errors are obtained when a later observed value such as x8 depends on those estimates.
  • Recurrent update: For irregular sampling, BRITS applies temporal decay to hidden states, with larger elapsed gaps producing smaller decay factors.The complement input replaces missing entries with estimated values before the recurrent update.
  • Core approach: Missing-value estimates remain variables in the RNN graph, allowing delayed forward and backward gradients with consistency constraints.This design is intended to improve estimation accuracy by fully propagating errors through imputed values.
  • Bidirectional imputation: Bidirectional dynamics provide a nearer backward error path, so missing estimates receive information from both later forward observations and closer preceding backward steps.The final estimation combines forward and backward predictions, with a consistency loss enforcing agreement between directions.
  • Correlated imputation: The correlated-feature variant combines history-based and feature-based estimates, weighting them using temporal decay and the observation mask.Feature-based estimates use other features at the same time, while the combined estimate fills missing entries for the next recurrent step.

5 Experiment

The experiments use three real-world datasets, standard imputation and task-specific classification metrics, and comparisons against statistical, matrix-based, and neural baselines.

  • Datasets: Experiments cover air quality, intensive-care health-care, and human-activity localization datasets.The air-quality data contain 13.3% missing values, the health-care data up to 78%, and the localization data include 30,917 time series.
  • Training: Classification models are pretrained for imputation and then optimized with 5-fold cross validation using imputation and classification losses jointly.For pure imputation, early stopping uses validation error from randomly selected non-missing values.
  • Evaluation: Imputation is evaluated with mean absolute error (MAE) and mean relative error (MRE).Air-quality evaluation uses original values, whereas health-care and activity evaluations use normalized values.
  • Evaluation: Classification uses AUC for imbalanced health-care outcomes and standard accuracy for relatively balanced activity labels.The health-care dataset contains 10% patients who died in hospital.
  • Baselines: The study compares four BRITS-family models implemented with PyTorch: RITS-I, RITS, BRITS-I, and BRITS.All models are trained with GPU GTX 1080.
  • Baselines: Baselines include mean imputation, KNN, matrix factorization, MICE, ImputeTS, STMVL, M-RNN, and GRU-D.STMVL is specific to air-quality imputation, while GRU-D is designed for health-care data.

5.3 Experiment Results

BRITS outperforms the compared baselines on both imputation and classification tasks, while bidirectional dynamics and feature correlations improve performance within the BRITS variants.

  • Imputation Results: BRITS significantly outperforms all baseline models on imputation tasks.ImputeTS performs best among non-RNN methods, while STMVL performs well on air-quality data but cannot be applied to the other datasets.
  • Imputation Results: Bidirectional recurrent dynamics and feature correlations both enhance performance among the four BRITS models.This conclusion follows from comparing the performances of the BRITS variants.
  • Classification Results: 0.850 ± 0.002 AUC and 0.969 ± 0.008 accuracy are reported for BRITS on the health-care and human-activity tasks, respectively.These results exceed the corresponding reported GRU-D values of 0.828 ± 0.004 and 0.939 ± 0.010.

6 Conclusion

The conclusion presents BRITS as a recurrent method that learns missing values directly without data-generating-process assumptions and jointly performs imputation with classification or regression.

  • Conclusion: BRITS imputes multivariate time-series missing values with a bidirectional recurrent dynamical system without specific assumptions over the data.Missing values are treated as variables in the bidirectional RNN graph, receiving delayed gradients in both directions.
  • Conclusion: BRITS jointly performs missing-value imputation and classification or regression within one neural network.The reported experiments show more accurate imputation and classification/regression than state-of-the-art methods.

A Performance Comparison for Classification Tasks

The classification comparison reports BRITS as outperforming the other RNN-based baselines and contrasts its training behavior with GRU-D.

  • Classification Comparison: BRITS outperforms the other RNN-based models on the classification tasks.The comparison includes GRU-D and M-RNN across health-care and human-activity classification.
  • Classification Comparison: BRITS uses imputation errors as supervised signals, whereas GRU-D requires p = 0.25 dropout on its top classification layer to prevent overfitting.The paper reports that dropout seems unnecessary for BRITS during training.

B Performance Comparison for Univariate Synthetic Data

Synthetic experiments compare ImputeTS with BRITS-I on univariate series containing about 22% missing values, with Figure 3 showing example imputations.

  • Figure 3 organizes three examples by time series, with ImputeTS imputations in the first row and BRITS-I imputations in the second.Each column corresponds to one time series.
  • BRITS-I demonstrates better imputation performance than ImputeTS in the displayed synthetic examples.The comparison is presented visually in Figure 3.
  • ImputeTS fails when the beginning of a time series is missing, whereas BRITS-I uses backpropagated errors to adjust early imputations.Delayed gradients from later observations make the start-part imputations more accurate.

C Performance for Non-differentiable ˆxt

The experiment compares BRITS with BRITS-cut, which stops gradients through imputed values, during health-care data imputation training.

  • BRITS-cut initially reduces validation error quickly but soon fails because of overfitting.The early advantage occurs during the first 20 iterations.
  • Treating imputed values as variables in the RNN graph allows them to be updated during backpropagation.The comparison evaluates this design against the non-differentiable BRITS-cut variant.

D Test Data of Air Quality Imputation

The air-quality imputation evaluation uses a month-based train/test split and identifies test timeslots by checking corresponding positions in the previous month.

  • The 3rd, 6th, 9th, and 12th months form the test set, while the remaining months form the training set.This follows the same selection method as prior work.
  • A test timeslot is selected by checking whether its corresponding position in the previous month is absent.For example, an observed measurement at 8 o’clock on 2015/03/07 is checked against 8 o’clock on 2015/02/07.
Loading 1805.10572v1…