PaperlayerView exact revision on arXiv

Source-linked AI summary

On Unlearning for Time-series Forecasting

Zeyu Shi, Yanhui Luo, Ziming Hong, Chongyang Gao, Kezhen Chen, Shanshan Ye, Lixu Wang

arXiv:2610.02865v1cs.LG

TL;DR

Time-series unlearning must remove training influence despite overlapping forecasting windows and the lack of natural replacement targets for continuous forecasts. RDTU uses a retained-set NTK prediction, structural support, and residual diffusion to construct labels for approximate unlearning. Across four benchmarks, it reports the lowest F-P Gap and test NRMSE among evaluated efficient methods, while maintaining low membership distinguishability and using substantially less computation than exact retraining.

  • Problem

    Overlapping forecasting windows and context-dependent continuous targets make it difficult to construct deletion targets that approximate exact-unlearning predictions for time-series models.

    Source: §1 · p. 2, §3.1 · p. 5

  • Method

    RDTU conditions residual diffusion on a retained-set NTK base prediction and SIS, then uses the residual-corrected labels to guide a lightweight unlearning update.

    Source: §1 · p. 2, §3.3 · p. 8

  • Results

    Across four forecasting benchmarks, RDTU achieves the lowest F-P Gap and test NRMSE among evaluated efficient unlearning methods while maintaining low membership distinguishability and requiring substantially less computation than exact retraining.

    Source: §5 · p. 10

  • Takeaways & Limitations

    RDTU provides an approximate, label-guided alternative to retraining for time-series unlearning, with performance evaluated across univariate and multivariate benchmarks.

    Source: §1 · p. 2, §5 · p. 10

    RDTU provides approximate rather than certified unlearning and is not a formal or legal guarantee of data erasure.

    Source: ETHICS STATEMENT · p. 11

Abstract

from arXiv · show

Time-series forecasting is widely used in sensitive domains. Models in these settings are often trained on longitudinal user- or entity-level records, which may later require removal because they contain sensitive or proprietary information or have been corrupted by sensor failures. To address such deletion requests without costly retraining, machine unlearning has been widely studied as a practical mechanism for privacy protection and data governance. However, the application of machine unlearning to time series prediction has not yet been well realized; this is mainly due to the following unique challenges: Gradient-based unlearning can be unstable because a deleted observation participates in multiple causally connected forecasting windows, causing parameter updates to propagate beyond the requested interval and degrade retained forecasting utility. Label-guided updating offers a more controlled alternative, but continuous and context-dependent forecasts lack a suitable replacement target, while the exact-retrained output is unavailable during unlearning. Moreover, the remaining support for a deleted temporal pattern is highly non-uniform. Some affected windows retain structurally similar counterparts in the retained data, whereas others become underrepresented or isolated. We present RDTU, a Residual Diffusion framework for time-series unlearning. RDTU first uses a retained-set neural tangent kernel predictor to obtain a deletion-compatible base forecast. Then it quantifies the global and local structural support of each affected window using the volume contribution of the retained-reference data. Then a diffusion model generates a residual correction that estimates the counterfactual forecast, yielding a pseudo-label field that guides a lightweight model update. Experiments show that RDTU consistently produces unlearned models that most closely match exact retraining.

1 INTRODUCTION

Forecasting models may need to forget sensitive or corrupted observations, but time-series unlearning must account for overlapping windows and context-dependent continuous targets. RDTU constructs counterfactual labels using a retained-set NTK predictor, structural support, and residual diffusion to guide approximate unlearning.

Motivation

  • Forecasting unlearning is motivated by selective removal needs in domains including finance and healthcare.Forecasting supports decisions across several domains using historical observations.Source: §1 · pp. 1–2
  • Forecasting observations may require deletion because they contain sensitive information or are corrupted by sensor and communication faults.Equipment aging can also make previously collected operating patterns obsolete.Source: §1 · p. 2

Unlearning approaches

  • Exact unlearning seeks to match retraining on retained data, but its computational cost motivates approximate methods that update the original model.Gradient-ascent approaches can be unstable, and label-based approaches construct alternative targets without specifying desired post-deletion behavior.Source: §1 · p. 2

Challenges

  • Time-series unlearning is challenging because overlapping windows share temporal representations, while continuous forecasts lack a natural replacement target.Existing Gaussian-input targets are not conditioned on the query or its local retained-data support and cannot approximate exact-unlearning predictions.Source: §1 · p. 2

Contributions

  • RDTU combines counterfactual labels, a retained-set NTK predictor, SIS, and residual diffusion to guide approximate time-series unlearning.SIS summarizes global directional change and local retained support, while diffusion corrects the NTK prediction gap.Source: §1 · p. 2

2 PROBLEM DEFINITION AND RELATED WORKS

The section defines forecasting samples as input-context and future-target windows, then formulates deletion and unlearning in terms of affected windows and retained data. It also reviews gradient and random-labeling approaches and introduces the NTK predictor as a linearized approximation whose predictions can differ from a trained finite-width network.

2.1 SLIDING-WINDOW FORECASTING

  • Each forecasting sample pairs a lookback input window with a future target window, and training minimizes empirical forecasting loss.The stated common loss is mean squared error between predicted and ground-truth future windows.Source: §2.1 · p. 3

2.2 TIME-SERIES EXACT UNLEARNING AND APPROXIMATE UNLEARNING

  • A deletion request affects windows whose temporal footprints overlap the requested interval, and exact unlearning retrains from scratch on the strictly retained set.Approximate unlearning instead updates the original model to make its forecasting behavior close to exact retraining.Source: §2.2 · p. 3
  • Gradient ascent on forgetting samples can destabilize optimization and degrade retained-set utility, while random labeling replaces their targets with random incorrect values.Random labeling weakens the original input-target relationship without retraining from scratch.Source: §2.2 · p. 4

2.3 NEURAL TANGENT KERNEL

  • The NTK predictor uses a regularized kernel regression on a supervised dataset, but practical finite-width neural predictions can differ from its approximation.The discrepancy is associated with nonlinear feature learning, stochastic optimization, and finite-kernel approximations.Source: §2.3 · p. 4

3 METHOD

RDTU constructs counterfactual labels for affected forecasting windows using a retained-set NTK predictor, structural support scores, and diffusion-generated residual corrections. The labels guide an update that aims to remove affected-window influence while preserving forecasting performance on retained windows.

3 METHOD

  • RDTU combines a retained-set NTK base forecast, SIS-based structural support valuation, and diffusion-generated residual corrections to guide unlearning updates.The generated label field is used to remove the influence of affected windows while preserving forecasting performance on strictly retained windows.Source: §3 · p. 5

3.1 NTK PREDICTOR BASED LABEL CONSTRUCTION

  • The NTK predictor efficiently approximates exact-retrained outputs on the strictly retained set, while diffusion models the residual gap between NTK and neural predictions.The method assumes residual patterns learned from the full training set transfer to retained-set conditions when deletion induces only a limited shift in temporal-pattern distribution.Source: §3.1 · pp. 5–6

3.2 STRUCTURAL IRREPLACEABILITY SCORE VALUATION

  • SIS combines global volume change with local nearest-neighbor support to characterize how poorly each affected window is supported by retained patterns.The global score measures temporal-pattern-space coverage, while the local score captures whether close-context counterparts exist.Source: §3.2 · p. 7

3.3 LABEL FIELD-GUIDED UNLEARNING

  • The update aligns affected-window predictions with generated labels and uses a separate retained-window batch as an anchoring signal for unaffected forecasting behavior.The affected-window batch supplies the forgetting signal; the retained batch stabilizes the update and preserves behavior on unaffected temporal patterns.Source: §3.3 · pp. 7–8

4 EXPERIMENTS

Across four public forecasting datasets, RDTU improves agreement with exact retraining and forecasting utility over the evaluated baselines. Ablations support residual correction, diffusion-based residual modeling, and SIS, while the diffusion-step analysis identifies a quality–runtime trade-off.

4.1 DATASETS AND EXPERIMENTAL SETUPS

  • Experiments cover four public datasets, with separate models for 10 series or subjects per dataset and three deletion seeds per model.Air Quality and Traffic are univariate; EEG and HAR70+ are multivariate. Evaluation includes U/N AUC, F-P Gap, test NRMSE, and runtime.Source: §4.1 · p. 8

4.2 OVERALL PERFORMANCE

  • RDTU improves F-P Gap by 16.3%–56.2% and NRMSE by 3.2%–18.0% over the best baselines, SCRUB-R and TS-Unlearn, across four datasets.Table 1 compares univariate and multivariate tasks; RDTU’s U/N AUC was not always optimal, though it retained an advantage over forgetting models without training breakdown.Source: §4.2 · p. 8, Table 1 · p. 9

4.3 ABLATION STUDY

  • Removing residual correction degrades label quality, deterministic MLP or ARIMA residuals perform worse than diffusion, and removing SIS increases F-P Gap by 3.1%–9.1%.The ablations attribute these results to the gap between the NTK predictor and forecasting model, the limits of deterministic regression for complex residual distributions, and SIS’s structural information.Source: §4.3 · p. 9, Table 2 · p. 9

4.4 HYPERPARAMETER ANALYSIS

  • Increasing diffusion steps from 16 to 128 reduces F-P Gap by 0.8%–11.2%, while increasing to 256 yields marginal changes of -0.01 to 0.09 percentage points and adds 0.39–0.70 seconds.The paper uses 128 steps as its default because further steps bring diminishing benefits at additional diffusion runtime.Source: §4.4 · pp. 9–10, Figure 2 · p. 9

5 CONCLUSION

RDTU combines an NTK base prediction, structural support characterization, and conditional residual diffusion to construct labels for a lightweight unlearning update. On four forecasting benchmarks, it achieves the lowest F-P Gap and test NRMSE among evaluated efficient methods while maintaining low membership distinguishability and using substantially less computation than exact retraining.

5 CONCLUSION

  • RDTU combines a retained-set NTK predictor, SIS-based characterization of affected-window support, and conditional residual diffusion to produce labels for a lightweight unlearning update.The residual diffusion corrects the discrepancy between the NTK approximation and the corresponding neural predictor.Source: §5 · p. 10
  • Across four univariate and multivariate benchmarks, RDTU achieves the lowest F-P Gap and test NRMSE among evaluated efficient unlearning methods while maintaining low membership distinguishability.Source: §5 · p. 10
  • RDTU requires substantially less computation than exact retraining.Source: §5 · p. 10

AI USE STATEMENT

The authors used generative AI solely for linguistic editing and report that it was not used for research design, experiments, analysis, interpretation, or scientific conclusions.

AI USE STATEMENT

  • Generative AI supported linguistic editing only; the authors state it was not used to formulate hypotheses, develop methods, conduct experiments, analyze data, interpret results, or generate conclusions.Source: AI USE STATEMENT · p. 11

ETHICS STATEMENT

The study uses publicly released benchmark data, including previously collected human-subject measurements, and makes no formal or legal guarantee of data erasure. The authors state that effectiveness depends on threat model, evaluation protocol, and deployment setting.

ETHICS STATEMENT

  • RDTU provides approximate rather than certified unlearning and is not a formal or legal guarantee of data erasure.Source: ETHICS STATEMENT · p. 11
  • Effectiveness depends on the adopted threat model, evaluation protocol, and deployment setting.The authors advise appropriate access controls, deletion audits, and regulatory review for privacy-sensitive applications.Source: ETHICS STATEMENT · p. 11
  • Experiments use existing public datasets, including previously collected EEG and HAR human-subject measurements; no new participants or personal data were collected.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

The experiments use four public datasets and a consistent deletion protocol, while RDTU constructs phase-specific structural support scores and uses conditional residual diffusion to generate counterfactual labels for model updating.

A.1 DATASETS

  • Experiments use four public datasets, evaluate 10 sequences or subjects per dataset under three deletion seeds, and macro-average across sequences or subjects.Deletion requests are contiguous raw-observation blocks; every forecasting example whose input or target overlaps a block is removed.Source: REPRODUCIBILITY STATEMENT · p. 11, §A.1 · p. 14

A.2 PHASE-SPECIFIC SIS CONSTRUCTION

  • SIS measures structural change relative to a phase-specific reference: the complete training set during diffusion training and the retained set during residual generation.The global component uses the absolute change in log-determinant volume; a nearest-neighbor measure complements it with local support.Source: §A.2 · pp. 14–15

A.3 CONDITIONAL RESIDUAL DIFFUSION

  • RDTU diffuses the discrepancy between a forecasting model and its NTK predictor, rather than diffusing forecasting targets directly.The NTK prediction supplies the base forecast, while the diffusion model estimates residual corrections that capture effects beyond the NTK approximation.Source: §A.3 · pp. 15–16
  • At unlearning time, RDTU replaces complete-data NTK predictions and SIS values with retained-set counterparts to condition residual generation.The denoiser is kept fixed, under the assumption that the learned conditional residual pattern transfers when the reference condition changes.Source: §A.3 · p. 16

A.4 END-TO-END UNLEARNING PROCEDURE

  • The generated residual is added to the retained-set NTK prediction to form guiding labels for updating the original model on the strict deletion closure.Diffusion training is performed once for the original model; each deletion request requires retained-condition construction, label generation, and model updating.Source: §A.3 · p. 16, §A.4 · p. 16

A.5 METRICS

The evaluation covers membership indistinguishability, agreement with exact retraining, forecasting utility, and runtime; it also defines temporal-unit auditing and compares RDTU with representative unlearning baselines.

A.5 METRICS

  • Unlearning effectiveness is assessed by two-sided U/N AUC and F-P Gap, while test NRMSE measures forecasting utility and wall-clock time measures computational efficiency.The F-P Gap is the normalized prediction difference from exact retraining on affected windows; a lower value means closer functional agreement.Source: §A.5 · pp. 16–18
  • Membership audits operate on temporal deletion units whose closure includes every forecasting window with input context or target overlapping the interval.The closure operator is C(T′) = {(x_t, y_t) | I_t ∩ T′ ≠ ∅}, with I_t = [t − ℓ, t + h).Source: §A.5 · p. 17
  • Two-sided U/N AUC reports membership distinguishability without depending on score direction, with values closer to 0.5 indicating harder distinction.The metric lies in [0.5, 1].Source: §A.5 · pp. 16, 18
  • Test NRMSE normalizes test RMSE by the original training-series standard deviation, whereas F-P Gap compares affected-window predictions with exact retraining.Lower values indicate better forecasting performance for NRMSE and closer functional agreement for F-P Gap.Source: §A.5 · p. 18

A.6 BASELINES

  • The comparison includes exact retraining and approximate baselines using parameter damping, gradient updates, replacement labels, teacher–student updates, or Pareto balancing.Retrain trains the same architecture from scratch on the strictly retained set and serves as the exact-unlearning reference.Source: §A.6 · p. 19

B MORE EXPERIMENTS

Across four datasets, RDTU remains effective at forecasting horizons of three and six steps. At horizon three, it leads approximate methods on F-P Gap and test NRMSE, while the diffusion-step trend supports using 128 steps.

B MORE EXPERIMENTS

  • At horizon 3, RDTU has the lowest F-P Gap among approximate methods on all four datasets, reducing it by 21.5%, 0.8%, 19.6%, and 13.2% on Air Quality, Traffic, EEG, and HAR70, respectively.The gains indicate counterfactual labels remain close to exact-retraining predictions for multi-step forecasts.Source: B MORE EXPERIMENTS · p. 19, Table 3 · p. 19, Table 4 · p. 20
  • At horizon 3, RDTU has the lowest test NRMSE among approximate methods across datasets, improving approximately 0.2%–2.5% over the best-performing baseline.Its U/N AUC values range from 0.5207 to 0.5483 without the severe forecasting degradation reported for several other methods.Source: B MORE EXPERIMENTS · p. 19
  • At horizon 6, the authors report that RDTU also achieves excellent results, with univariate and multivariate comparisons presented in Tables 5 and 6.The supplied passages do not give the table values.Source: B MORE EXPERIMENTS · p. 20, Table 5 · p. 20

C.1 INFLUENCE OF DIFFUSION STEPS ON DIFFERENT HORIZON LENGTH

  • For horizon length 3, the influence of diffusion-step count on model performance and training duration follows the same trend as horizon length 1, so the authors set the diffusion-step count to 128.Source: §C.1 · p. 20

C.2 INFLUENCE OF FORGET LOSS WEIGHT

The preferred forgetting-to-retention loss balance differs across datasets, while the authors identify computational costs that may grow with sequence length, data dimensionality, and model size.

C.2 INFLUENCE OF FORGET LOSS WEIGHT

  • The selected λf/λr ratios are 1.8, 0.8, 0.2, and 0.3 for Air Quality, Traffic, EEG, and HAR70, respectively.The authors report moderate test NRMSE changes at these choices and attribute the differing preferred balances to dataset temporal structures and prediction scales.Source: §C.2 · pp. 20–21

D.1 COMPUTATIONAL SCALABILITY

  • RDTU’s computational cost may increase for long sequences, high-dimensional multivariate data, and large forecasting architectures.Empirical NTK construction requires parameter-gradient features for many windows, and residual generation uses multiple denoising steps per affected query.Source: §D.1 · p. 21
  • The authors identify incremental NTK and retained-reference updates, cached features, and accelerated or adaptive diffusion sampling as possible cost-reduction directions.They also call for scalability studies reporting preprocessing costs, per-request latency, peak memory, and amortized runtime across workload dimensions.Source: §D.1 · pp. 21–22

D.2 BROADER EVALUATION AND STRONGER GUARANTEES

The discussion identifies untested forecasting and deletion settings and states that current empirical checks do not formally guarantee data removal. It calls for broader model evaluations, stronger attacks, and theoretical error bounds.

D.2 BROADER EVALUATION AND STRONGER GUARANTEES

  • Current evaluation measures agreement with exact retraining and membership-inference resistance, but provides no formal guarantee of data removal.The authors suggest adaptive black-box and white-box, reconstruction, and model-inversion attacks, alongside theoretical bounds on approximation, residual-generation, and optimization errors.Source: §D.2 · p. 22
  • Future evaluations should cover longer horizons, higher-dimensional series, irregular sampling, missing observations, distribution shifts, and rare or rapidly changing deleted patterns.Source: §D.2 · p. 22
  • The residual-transfer assumption should be tested across deletion ratios, interval lengths, and numbers of sequential deletion requests.Source: §D.2 · p. 22
  • Evaluations with recurrent, convolutional, Transformer, and state-space models would further assess RDTU’s architecture-agnostic design.Source: §D.2 · p. 22

Related papers

Loading 2610.02865v1…