PaperlayerView exact revision on arXiv

Source-linked AI summary

Glocal Information Bottleneck for Time Series Imputation

Jie Yang, Kexin Zhang, Guibin Zhang, Philip S. Yu, Kaize Ding

arXiv:2510.04910v1cs.LGcs.AI

TL;DR

Under high missingness, reconstruction-focused time-series imputation objectives offer little guidance for learning global structure. Glocal-IB adds a global-alignment loss to the information bottleneck framework, and experiments on nine datasets show improved imputation accuracy and more stable latent representations.

  • Problem

    Under high missing rates, reconstruction objectives emphasize local numerical accuracy and provide little guidance for learning global time-series structure.

    Source: §3.3 · p. 6

  • Method

    Glocal-IB extends the information bottleneck framework with a tractable-mutual-information-based Global Alignment loss that aligns masked-input representations with those of fully observed inputs.

    Source: §5 · p. 10

  • Results

    Across nine datasets, Glocal-IB consistently improves imputation accuracy and produces more stable latent representation distributions under varying missing rates.

    Source: §5 · p. 10

  • Takeaways & Limitations

    Reported benefits extend beyond a vanilla Transformer to other backbones and include better robustness under high missing rates.

    Source: §1 · p. 3

    The local-loss derivation assumes time-series data follow a Gaussian distribution with fixed variance.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

Time Series Imputation (TSI), which aims to recover missing values in temporal data, remains a fundamental challenge due to the complex and often high-rate missingness in real-world scenarios. Existing models typically optimize the point-wise reconstruction loss, focusing on recovering numerical values (local information). However, we observe that under high missing rates, these models still perform well in the training phase yet produce poor imputations and distorted latent representation distributions (global information) in the inference phase. This reveals a critical optimization dilemma: current objectives lack global guidance, leading models to overfit local noise and fail to capture global information of the data. To address this issue, we propose a new training paradigm, Glocal Information Bottleneck (Glocal-IB). Glocal-IB is model-agnostic and extends the standard IB framework by introducing a Global Alignment loss, derived from a tractable mutual information approximation. This loss aligns the latent representations of masked inputs with those of their originally observed counterparts. It helps the model retain global structure and local details while suppressing noise caused by missing values, giving rise to better generalization under high missingness. Extensive experiments on nine datasets confirm that Glocal-IB leads to consistently improved performance and aligned latent representations under missingness. Our code implementation is available in https://github.com/Muyiiiii/NeurIPS-25-Glocal-IB.

1 Introduction

The introduction identifies a mismatch between local reconstruction objectives and the global structure needed for reliable time series imputation under high missingness. It proposes Glocal-IB, which adds latent-space alignment to preserve both global and local information.

Introduction

  • Under high missing rates, low training loss can coexist with substantially worse inference-time imputation; reducing training epochs can improve imputation despite slightly higher training loss.The authors interpret this as evidence that the objective encourages memorizing local observations rather than learning generalizable global structure.Source: §1 · p. 2
  • Better imputation corresponds to latent distributions aligned with those from fully observed data, while increasing missingness distorts representations despite low training losses.The authors observe that this distortion correlates with poor imputation and argue that MAE/MSE reconstruction losses fail to preserve globally coherent structure under severe missingness.Source: §1 · p. 2
  • Existing IB-based TSI methods [14] still rely on local reconstruction, and GPVAE [14] reaches MAE 0.8, similar to TimesNet [60] at the same missing rate.The introduction argues that timestamp-level accuracy can leave models sensitive to noise and unable to capture the underlying global distribution.Source: §1 · p. 2
  • Glocal-IB adds a Global Alignment loss, derived from a tractable mutual information approximation, to align masked-input representations with those of their corresponding original inputs.The alignment uses a single MLP, making the training paradigm model-agnostic and integrable into encoder-decoder frameworks.Source: §1 · p. 3
  • Glocal-IB combines lightweight global alignment with local reconstruction to enforce latent-space consistency, improve global and local feature learning, and remove irrelevant noise.The motivation arises because missing values act as structured noise that disrupts temporal dependencies and statistical patterns, while TSI methods train by randomly masking observed values.Source: §1 · pp. 1, 3
  • Across nine benchmark datasets, Glocal-IB consistently achieves top imputation performance, with similar improvements across backbones and greater robustness under high missing rates.When applied to a vanilla Transformer, it also helps form a smooth, structured latent space.Source: §1 · p. 3

2 Related Work

Prior TSI methods span multiple architectures but remain vulnerable to noise and temporal redundancy under point-wise reconstruction objectives. Glocal-IB responds with latent alignment intended to preserve semantic information without relying on foundation models.

Time Series Imputation

  • Recent TSI methods use encoder-decoder architectures spanning RNNs, [3], transformers [36], GNNs [8], [31], and probabilistic models [46], [14],,.These approaches differ mainly in how they capture temporal dependencies.Source: §2 · p. 3
  • Despite architectural progress, most TSI models remain sensitive to noise and temporal redundancy because they rely on point-wise reconstruction loss.Glocal-IB instead emphasizes both global structure and local detail to encourage semantically stable representations and improve imputation under severe missingness.Source: §2 · p. 3

Optimization Dilemma

  • In computer vision, VA-VAE [67] and REPA [70] align latent spaces using vision foundation models [39] [15] [16], whereas time-series foundation models [45] [54] lack sufficient semantic information.The cited work frames this gap as part of a mismatch between low training loss and poor test-time performance.Source: §2 · p. 3
  • Glocal-IB's Global Alignment objective keeps masked-sequence representations close to those of original observed sequences, using only one extra MLP.The approach is presented as lightweight and efficient compared with solutions that rely on foundation models.Source: §2 · p. 3

3 Methodology

For time series imputation, Glocal-IB uses an information bottleneck to regularize representations of masked inputs while retaining local detail and global structure. Its training objective combines regularization, reconstruction, and global alignment.

3.1 Overview

  • Glocal-IB balances noise reduction against retaining local details and global semantic features by minimizing I(Z; Xo) and maximizing I(Z; X).The framework is designed to align representations with the original data distribution and is model-agnostic, adding one MLP projector.Source: §3.1 · pp. 4–5

3.2 Regularizing Partially Observed Input: min Iθ(Z; Xo)

  • The regularization term upper-bounds I(Z; Xo) with an expected KL divergence between the encoder posterior and a prior, yielding an analytically differentiable loss.The latent prior is an isotropic Gaussian, and the encoder models partially observed series with a multivariate Gaussian.Source: §3.2 · p. 5

3.3 Maximizing Global and Local Inforamtion: max Iϕ(X; Z)

  • Global alignment uses an InfoNCE-inspired bound to match masked-input representations with original-data embeddings at the same timestamp, using other timestamps as negatives.A one-layer MLP projector and the encoder define the density ratio; the alignment is intended to capture temporal dynamics and global data distribution. By contrast, point-wise MSE can memorize noisy details under high missingness, degrading imputations and distorting latent representations.Source: §3.3 · p. 6

3.4 Overall Training Objective

  • The overall objective combines regularization, local reconstruction, and global alignment losses, with hyperparameters balancing their mutual-information contributions.The framework applies to encoder-decoder architectures and requires only one additional MLP.Source: §3.4 · p. 6

4 Experiments

Across nine datasets, Glocal-IB improves imputation performance, preserves coherent latent structure under high missingness, and remains effective across missing patterns with modest computational overhead. Experiments also find that its alignment and regularization components contribute to performance, while excessive regularization can hurt it.

4.1 Experimental Settings

  • Experiments evaluate a two-layer Transformer with Glocal-IB against baselines on nine public datasets using MAE and MSE under point-wise random masking.The study follows PyPOTS train/validation/test splits and reports average performance across missing rates from 10% to 90%.Source: §4.1 · pp. 6–7, Table 1 · p. 7

4.2 Overall Comparison

  • Glocal-IB achieves the lowest MAE and MSE across all nine datasets, with MSE reductions of up to 40% on ETTh1, ETTh2, ETTm1, and ETTm2 compared with all baselines.It also surpasses SAITS and TimesNet on several challenging real-world datasets and outperforms the second-best method on Weather.Source: §4.2 · p. 7, Table 1 · p. 7
  • Glocal-IB preserves coherent latent structure from 10% to 70% missingness and retains the original distribution’s shape at 90%, while unaligned models collapse or fragment.The reported visualizations cover the distributions of SAITS, TimesNet, and Glocal-IB, and show well-structured Glocal-IB representations up to 90% missingness.Source: §4.2 · pp. 7–8, Figure 4 · p. 8, §4.3 · p. 9

4.3 Generality Analysis

  • From 10% to 70% missingness, Glo_1 and Glo_2 improve TimesNet and SAITS over original and Time-MoE-aligned training, and still help Transformer and SAITS at 90%.Time-MoE alignment yields only marginal gains, which the authors attribute to forecasting-focused pretraining that may not provide sufficient semantic constraints for imputation.Source: §4.3 · pp. 8–9

4.4 Missing Pattern and Efficiency Analysis

  • Glocal-IB achieves the lowest MAE on block-wise missing data and improves existing models across point and block missing patterns, especially for block missingness.The reported ETTh2 comparison includes 50% point and block missing rates; additional missing-rate results are in Appendix D.3.Source: §4.4 · p. 9, Figure 5 · p. 9
  • Adding Glocal-IB causes only marginal increases in memory use and execution time, unlike the substantial overhead of Time-MoE foundation-model alignment.The efficiency comparison evaluates four representative models on ETTh1.Source: §4.4 · p. 9, Figure 5 · p. 9

4.5 Ablation Study and Sensitivity Analysis

  • Ablations show that removing Global Alignment worsens performance more as missingness rises, while regularization suppresses irrelevant latent variation and Global Alignment preserves global semantic information.The study compares the full method with variants removing regularization or Global Alignment and with a local-loss-only variant.Source: §4.5 · p. 10
  • Increasing the Global Alignment loss weight yields stable performance trends, but a Regularization loss weight above 0.01 sharply degrades imputation quality.A small amount of KL regularization helps filter noise, whereas excessive regularization suppresses useful latent information.Source: §4.5 · p. 10

5 Conclusion

Glocal-IB addresses an optimization dilemma in time-series imputation by adding a Global Alignment loss to standard IB-based objectives. Experiments on nine datasets show improved imputation accuracy and more stable latent representation distributions across varying missing rates.

5 Conclusion

  • Across nine datasets, Glocal-IB consistently improves imputation accuracy and stabilizes latent representation distributions under varying missing rates.The conclusion reports these gains across the evaluated datasets without specifying numerical results.Source: §5 · p. 10
  • Glocal-IB adds a Global Alignment loss based on a tractable mutual information approximation to standard IB-based objectives.The loss encourages masked-input representations to match those of fully observed counterparts, retaining global structure and local detail while reducing noise.Source: §5 · p. 10
  • Glocal-IB is introduced to address the optimization dilemma in current time-series imputation methods.Source: §5 · p. 10

A Theoretical Analysis … A.2.2 Derivation of Lϕ

The theoretical derivations use variational bounds to make the mutual-information objectives tractable, then express the global alignment objective as a contrastive approximation to I(X; Z). The derivations specify how variational distributions and in-batch negatives support these objectives.

A.1 Variational Approximation of Mutual Information Minimization for LθReg

  • The I(Z; Xo) minimization is approximated by a variational upper bound, following prior work [1].The derivation rewrites mutual information in KL form and introduces a variational marginal q(z) to make the expression tractable.Source: §A.1 · p. 15
  • The upper-bound derivation uses KL divergences between p(z|xo) and q(z), with non-negativity supporting the final inequality.The displayed expectation averages DKL[p(z|xo)||q(z)] over p(xo).Source: §A.1 · p. 15

A.2.1 Derivation of Lϕ

  • The derivation of I(X; Z) uses variational inference to obtain a lower bound, with the reconstruction conditional approximated by qϕ(x|z).The displayed KL term compares p(x|z) with qϕ(x|z), averaged over p(z).Source: §A.2.1 · pp. 15–16
  • Non-negativity of the KL divergence supports the lower-bound step in the I(X; Z) derivation.The derivation identifies the discrepancy between p(x|z) and qϕ(x|z) as a KL divergence.Source: §A.2.1 · p. 16

A.2.2 Derivation of Lϕ

  • For global-level guidance, I(X; Z) is approximated in a contrastive form similar to CPC [38].The mini-batch approach chooses Xneg from data at other timestamps in the same mini-batch, and f(x, z) is proportional to p(x|z).Source: §A.2.2 · p. 16
  • The simplified approximation is I(X; Z) ≈ −E_p(x,z)[f(x, z)], yielding the alignment loss in Eq. 13.This simplification is motivated by developments in contrastive learning [15] [5].Source: §A.2.2 · p. 16

B Datasets

The experiments use nine real-world time-series datasets spanning energy, air quality, traffic, and weather measurements.

B Datasets

  • ETT [75] includes four transformer subsets, sampled hourly or every 15 minutes, while Beijing Air combines 11 variables from 12 stations into 132 hourly features.ETT records seven power-related factors from 2016/07 to 2018/07; Beijing Air spans 2013/03/01 to 2017/02/08.Source: B Datasets · p. 17
  • PEMS-Traffic [59] contains hourly occupancy rates from 862 San Francisco Bay sensors, and Electricity [59] records hourly usage from 321 clients.PEMS-Traffic spans 2015/01 to 2016/02, while Electricity covers 2012–2014.Source: B Datasets · p. 17
  • Weather [59] provides 21 variables sampled every 10 minutes in 2021, while Metr-LA [26] records five-minute speeds from 207 Los Angeles sensors.Weather measurements come from the Max Planck Biogeochemistry Institute; Metr-LA covers 2012/03 to 2012/06.Source: B Datasets · p. 17

C Implementation Details · D Full Experiments · D.1 Full Comparison Results

The experiments use a fixed chronological data protocol and PyPOTS-based baselines, with results averaged across five seeds. The full comparison covers five missing rates and includes a latent-space visualization, but the supplied passage does not report the comparison outcomes.

C Implementation Details

  • The data follow the PyPOTS protocol, with chronological 60%/20%/20% training, validation, and test splits to avoid leakage.Input sequence length is 96 for all datasets.Source: C Implementation Details · p. 17
  • Experiments use PyTorch 2.6.0 on one NVIDIA 4090 GPU with 24GB memory.The passage specifies the software and hardware environment.Source: C Implementation Details · p. 17
  • Training uses Adam with a learning rate of 0.001, batch size 64, 30 epochs, and hidden dimension 256.Source: C Implementation Details · p. 17
  • Baseline models are built on the PyPOTS benchmark and follow their original papers’ and official implementations’ settings.Reported results are averages over five random seeds.Source: C Implementation Details · p. 17

D.1 Full Comparison Results

  • Table 2 compares Glocal-IB with a vanilla Transformer against baseline methods across missing rates of 10%, 30%, 50%, 70%, and 90%.The passage identifies the comparison scope but does not provide table values or establish a winner.Source: §D.1 · p. 17
  • Figure 8 visualizes the latent spaces of Glocal-IB and three representative time-series imputation methods.The passage describes the visualization but does not report its specific patterns.Source: §D.1 · p. 17

D.2 Generality Analysis

Glocal-IB improves latent-space alignment for TimesNet and SAITS across missing rates from 10% to 90%.

D.2 Generality Analysis

  • Across missing rates of 10%, 30%, 50%, 70%, and 90%, Glocal-IB improves latent-space alignment for TimesNet and SAITS.Fig. 9 shows the latent spaces of both models and their Glocal-IB counterparts.Source: §D.2 · p. 17

D.3 Missing Pattern Analysis · D.4 Ablation Study and Sensitivity Analysis · E Societal Impact Statement

Across four ETT datasets and missing rates from 10% to 90%, Glocal-IB achieves the best imputation performance. The societal impact statement notes potential healthcare benefits alongside privacy and decision-making risks, and calls for safeguards.

D.3 Missing Pattern Analysis

  • Glocal-IB achieves the best imputation performance across ETTh1, ETTh2, ETTm1, and ETTm2 at missing rates from 10% to 90%.The comparison is presented in Figures 10–13.Source: §D.3 · p. 17

D.4 Ablation Study and Sensitivity Analysis

  • Ablation studies compare model variants on ETTh1, ETTh2, ETTm1, and ETTm2.The studies are presented in Figure 15; the supplied passage does not report their outcomes.Source: §D.4 · p. 17
  • Parameter sensitivity analyses cover ETTh1, ETTh2, ETTm1, and ETTm2.The analyses are presented in Figure 14; the supplied passage does not report their outcomes.Source: §D.4 · p. 18

E Societal Impact Statement

  • Following previous TSI works [3], the paper notes that improved imputation could support more reliable patient monitoring, earlier diagnosis, timely interventions, and lower medical costs.Recovered clinical measurements may inform healthcare decision-making.Source: E Societal Impact Statement · p. 18
  • In surveillance and other sensitive settings, reconstructing incomplete data may raise privacy concerns, while over-reliance on imputation may cause errors to go unnoticed and downstream decisions to be biased or unreliable.Source: E Societal Impact Statement · p. 18
  • Responsible deployment calls for data-protection rules, transparency, fairness-aware validation, broader access, and regular audits to reduce unintended harm.Source: E Societal Impact Statement · p. 18

F Limitation and Discussion

Glocal-IB is evaluated on a limited set of backbones and ETT datasets, and its gains are less pronounced at extreme missingness. The discussion points to stronger structural priors or pretrained knowledge as possible ways to improve global guidance when observations are scarce.

F Limitation and Discussion

  • Computational constraints limited Glocal-IB evaluation to TimesNet, SAITS, and Transformer on ETTh1, ETTh2, ETTm1, and ETTm2, leaving broader models and datasets for future validation.The authors describe the results as sufficient to demonstrate effectiveness but call for broader evaluation to further validate generalization.Source: F Limitation and Discussion · p. 18
  • At 90% missingness, performance gains are less pronounced than at 10%–70%, possibly because 10% observed input provides too weak a global structure for meaningful alignment.With weak alignment signals, the model has limited capacity to distinguish signal from noise, resulting in less reliable imputation.Source: F Limitation and Discussion · p. 18
  • A future direction is to strengthen global guidance under limited observations, for example by incorporating stronger structural priors or pretrained knowledge into the latent space.Source: F Limitation and Discussion · p. 18
Loading 2510.04910v1…