PaperlayerView exact revision on arXiv

Source-linked AI summary

Energy Time-Series Imputation with Differentially Private Diffusion Models via Clipping-Aware Objective Conditioning

Huizhen Huang, Yu Li, Tao Huang, Chen Hou

arXiv:2610.00209v1cs.LG

TL;DR

Energy time-series imputation involves sensitive records, while late low-SNR diffusion timesteps can make ε-prediction gradients vulnerable to fixed-threshold DP-SGD clipping. The paper develops timestep-aware objective conditioning and reports improved utility over ε-prediction across evaluated energy recovery scenarios, alongside improved clipping diagnostics.

  • Problem

    Under cosine schedules, late low-SNR timesteps can produce large ε-prediction gradients before DP-SGD clipping, while this objective–clipping interaction in energy imputation remains insufficiently characterized.

    Source: §1 · p. 2, §2.1 · p. 5

  • Method

    The method combines v-prediction, static loss weighting as a uniform-scaling control, and schedule-aware dynamic weighting for diffusion-based energy imputation under fixed-threshold DP-SGD.

    Source: §1 · pp. 2–3

  • Results

    The method improves over standard ε-prediction across random point, contiguous block, and persistent outage missingness, with gains retained across evaluated severities and suffix-missing forecasting.

    Source: §5 · p. 24

  • Takeaways & Limitations

    The results show clipping-aware objective conditioning can improve both clipping behavior and interpolation utility under fixed-threshold DP-SGD.

    Source: §5 · p. 24

    The evaluation uses a cosine schedule, fixed diffusion backbone, matched DP-SGD configurations, and a window-level privacy unit; dynamic-weighting gains vary by configuration and task.

    Source: §6 · p. 24

Abstract

from arXiv · show

Reliable recovery of missing measurements is important for monitoring and analysis in energy time-series systems, where fine-grained measurements may contain sensitive temporal information. Diffusion models trained with differentially private stochastic gradient descent (DP-SGD) provide a promising framework for privacy-sensitive energy time-series imputation. Under cosine diffusion schedules, late timesteps correspond to low signal-to-noise ratio (SNR) conditions, where standard $\varepsilon$-prediction can induce large pre-clipping gradients. Such gradients are more likely to be clipped, reducing the retained optimization signal. The artificial intelligence (AI) contribution lies in formulating this objective--clipping interaction as an objective optimization problem under fixed-threshold DP-SGD and developing timestep-aware objective conditioning for diffusion-based energy time-series imputation. The method adopts $v$-prediction to mitigate late-timestep gradient amplification, uses static loss weighting as a uniform-scaling control, and introduces diffusion-schedule-aware dynamic weighting for stronger attenuation before clipping. For the engineering application, we evaluate the method on five real-world energy time-series datasets across random point missingness, contiguous block missingness, persistent outages, and multiple missing-data severities. Under matched DP-SGD settings, the proposed method consistently improves imputation utility over the $\varepsilon$-prediction baseline. Gradient diagnostics reveal lower upper-tail pre-clipping gradient norms, reduced clipping fractions, and stronger attenuation at late low-SNR timesteps, supporting the effectiveness of clipping-aware objective conditioning for energy time-series imputation.

1. Introduction

The introduction frames late low-SNR diffusion timesteps as a point where ε-prediction can produce gradients that are heavily attenuated by fixed-threshold DP-SGD clipping. It presents timestep-aware objective conditioning and evaluates its utility and gradient behavior across energy time-series recovery scenarios.

1. Introduction

  • At late low-SNR timesteps under cosine schedules, ε-prediction can produce gradients that exceed the fixed clipping threshold and lose optimization signal.This motivates conditioning the objective before DP-SGD clipping.Source: §1 · p. 2
  • The method combines v-prediction, static loss weighting as a uniform-scaling control, and schedule-aware dynamic weighting that attenuates the objective more strongly at late low-SNR timesteps.The clipping-and-noise mechanism, sampling rule, and diffusion backbone remain fixed across objective variants.Source: §1 · pp. 3–4
  • Under matched DP-SGD, the method improves imputation utility over standard ε-prediction across five energy datasets and multiple missingness scenarios.Gradient diagnostics also show lower high-percentile pre-clipping gradient norms and reduced overall clipping fractions.Source: §1 · pp. 3–4
  • The study characterizes how deterministic timestep weighting changes pre-clipping gradient norms and the effective clipping boundary on the unweighted-gradient scale.A controlled target–weighting decomposition separates target reparameterization effects from timestep-aware dynamic weighting.Source: §1 · p. 4

2. Related Work

Related work establishes private diffusion training, diffusion-based time-series recovery, and privacy-aware energy analysis as separate research directions. Their intersection—objective effects on DP-SGD clipping and energy recovery utility—remains insufficiently characterized.

2.1. Private Diffusion Optimization

  • Prior private diffusion studies establish feasibility across generative and adaptation settings, while the late-timestep objective–clipping interaction remains under-investigated.Under cosine schedules, late timesteps have low SNR, where ε-prediction can produce large pre-clipping gradients.Source: §2.1 · pp. 4–5

2.2. Objective Parameterization and Weighting

  • Time-series diffusion objectives include ε- and v-prediction and timestep weighting, but their effects on pre-clipping gradients under fixed-threshold DP-SGD remain uncharacterized.Existing conditional diffusion methods reconstruct missing temporal observations, while non-diffusion architectures provide recovery-performance references.Source: §2.2 · p. 5

2.3. Energy Time-Series Interpolation and Privacy

  • Energy records can reveal household activities and occupancy, yet the interaction between diffusion objective optimization and DP-SGD clipping in energy recovery has received limited attention.Prior work spans energy recovery, privacy risks, and privacy-preserving mechanisms, motivating study of their intersection.Source: §2.3 · p. 6

3. Method

The method frames late-timestep gradient amplification under fixed-threshold DP-SGD clipping as an objective-design problem. It combines target reparameterization and schedule-aware weighting, then characterizes how deterministic weighting changes pre-clipping gradients and clipping behavior.

3.1. Preliminaries

  • Masked-value interpolation uses time-series windows as independent DP-SGD records, with privacy defined at the window level and no cross-window aggregation.The diffusion process uses a cosine schedule, whose retained signal coefficient decreases at late timesteps.Source: §3.1 · pp. 6–7
  • At late low-SNR timesteps, standard ε-prediction can produce large pre-clipping gradients that clipping attenuates, reducing retained optimization signal.The method therefore conditions the denoising objective before fixed-threshold clipping.Source: §3.1 · p. 7

3.2. Our Method: Timestep-Aware Objective Conditioning

  • The method replaces ε-prediction with v-prediction and adds deterministic, schedule-aware weighting that more strongly attenuates late low-SNR timesteps.Static loss weighting serves as a uniform-scaling control; the dynamic weight is λt = ᾱt and depends on the diffusion schedule, not private data, gradients, or noise.Source: §3.2 · pp. 8–9

3.3. Theoretical Analysis

  • Deterministic timestep weighting scales the pre-clipping gradient norm; static weighting scales uniformly, while λt = ᾱt attenuates late low-SNR timesteps more strongly.The clipping threshold, noise multiplier, and sampling rule remain fixed in the analysis.Source: §3.3 · pp. 9–10
  • Weighting raises the effective clipping boundary on the unweighted-gradient scale, with the strongest increase at late timesteps where ᾱt is smallest.The actual DP-SGD clipping threshold remains fixed.Source: §3.3 · pp. 10–11
  • Clipping distortion depends on how much a gradient norm exceeds the threshold, motivating high-percentile pre-clipping norms and overall clipping fractions as diagnostics.The analysis is local to a fixed model state, record, and diffusion timestep; it does not characterize global training dynamics across independently trained variants.Source: §3.3 · p. 11

4. Experiments

The experiments compare objective variants under matched DP-SGD settings across five real-world energy time-series datasets, multiple missingness patterns and severities, and suffix-missing forecasting. They report improved utility over ε-prediction alongside gradient diagnostics, robustness tests, and comparisons with representative baselines.

4.2. Main Results

  • At 50% missingness, v-pred., dynamic achieves lower MSE, MAE, and RMSE than ε-prediction in all 15 dataset–pattern combinations.The comparison covers random point, contiguous block, and persistent outage missingness under matched DP-SGD settings.Source: Table 1 · p. 13, §4.2 · p. 13

4.3.1. Progressive Objective Ablation

  • Unweighted v-prediction supplies the main utility gain; static weighting varies and can worsen error, whereas dynamic weighting yields the lowest MSE and MAE in all nine dataset–pattern conditions.The progressive ablation attributes additional improvement to dynamic weighting over unweighted v-prediction and static controls.Source: §4.3.1 · pp. 14–15, Table 2 · p. 14

4.3.2. Controlled Target–Weighting Decomposition

  • Replacing ε-prediction with unweighted v-prediction reduces mean MSE 91.32%–99.13% and mean MAE 75.14%–90.95%; dynamic weighting then reduces them 8.69%–50.09% and 12.05%–34.02%.Dynamic weighting applied directly to ε-prediction has mixed effects across datasets.Source: §4.3.2 · pp. 15–16, Table 3 · p. 15

4.3.3. Gradient Diagnostics and Mechanism Analysis

  • Figure 2 shows strongest objective-induced gradient attenuation at late low-SNR timesteps; training diagnostics show dynamic weighting lowers clipping fractions from 1.000 to 0.944, 0.948, and 0.917 on ETTh1, ETTm1, and Appliances.The overall fractions remain above 91%; controlled diagnostics report median attenuation of about 0.8%–1.1% at t = 0–99 and 98.5%–99.1% at t = 900–999.Source: §4.3.3 · pp. 16–18, Figure 2 · p. 17, Table 4 · p. 18

4.4. Robustness to Missing Severity

  • Across five datasets and missing ratios from 10% to 50%, v-pred., dynamic has the lowest average MAE in all 20 dataset–severity combinations.Its MAE advantage over ε-prediction and unweighted v-prediction persists throughout the evaluated range.Source: Table 5 · p. 19, §4.4 · p. 19

4.5. Extension to Suffix-Missing Forecasting

  • In suffix-missing forecasting, v-pred., dynamic achieves the lowest MSE and MAE in all 15 dataset–horizon combinations at H = 24, 48, and 72.Both v-prediction variants outperform ε-prediction across five datasets and the three horizons.Source: Table 6 · p. 20, §4.5 · pp. 20–21

4.6.1. Comparison with Representative Time-Series Baselines

  • Under matched DP-SGD configurations, the proposed method achieves the lowest MSE and MAE across all six dataset–configuration combinations, outperforming TimeDiT ε-prediction, PatchTST, TimesNet, and iTransformer.The benchmark uses 50% random point missingness on ETTh1, ETTm1, and Electricity at ε = 8 and ε = 10.Source: §4.6.1 · p. 21, Table 7 · p. 22

4.6.2. Tuned ε-Prediction Baseline

  • After validation tuning strengthens ε-prediction, unweighted v-prediction retains the lowest MAE and RMSE in every reported dataset–condition comparison.Tuned ε-prediction reduces both metrics relative to the original baseline in all 12 comparisons; unweighted v-prediction's advantage remains across three interpolation patterns and suffix-missing forecasting.Source: §4.6.2 · pp. 22–23, Table 8 · p. 23

5. Conclusion

The study proposes timestep-aware objective conditioning for private diffusion-based energy time-series imputation, with target reparameterization driving the main utility improvement and dynamic weighting adding a gain under the main matched configuration. Gradient diagnostics support lower clipping-related gradient magnitudes, while evaluations show improvements across missingness patterns and settings.

5. Conclusion

  • The method progressively combines v-prediction, static loss weighting, and late-timestep dynamic attenuation while leaving the DP-SGD mechanism unchanged.It targets the objective–clipping interaction at late low-SNR timesteps, where ε-prediction can produce large pre-clipping gradients.Source: §5 · pp. 23–24
  • Target reparameterization is the primary source of utility improvement, while timestep-aware dynamic weighting adds a gain under the main matched DP-SGD configuration.The controlled gradient diagnostic shows stronger attenuation toward late low-SNR timesteps; training-time diagnostics show lower high-percentile pre-clipping norms and clipping fraction.Source: §5 · p. 24
  • Across five datasets, the method improves over ε-prediction across random-point, contiguous-block, and persistent-outage missingness under the main matched DP-SGD configuration.Improvements are retained across evaluated missing-data severities and suffix-missing forecasting.Source: §5 · p. 24

6. Limitations

The evaluation is bounded to a cosine schedule, fixed diffusion backbone, and the matched DP-SGD configurations studied; dynamic weighting’s incremental gain varies by configuration and task. The appendices provide implementation, evaluation, variability, comparison, and diagnostic details.

6. Limitations

  • The evaluation uses a cosine diffusion schedule, a fixed backbone, and the matched DP-SGD configurations considered in the study; the window-level privacy unit treats non-overlapping windows as independent records.Future evaluation should consider additional backbones and schedules, alternative private optimization settings, broader weighting strategies, and dedicated imputation baselines.Source: §6 · p. 24
  • Dynamic weighting’s incremental gain varies with training configuration and task setting.Appendix materials cover preprocessing and evaluation protocols, seed variability, baseline comparisons, full interpolation results, weighting comparisons, configuration sensitivity, and controlled gradient diagnostics.Source: §6 · p. 24, Appendix A · p. 25

Appendix A. Roadmap

  • Appendix A points readers to implementation and evaluation details, seed variability, baseline comparisons, full results, weighting comparisons, and gradient diagnostics.The roadmap says these materials support the main text without introducing new claims.Source: Appendix A · p. 25

Appendix B. Implementation Details

The implementation uses chronological splits, training-only normalization, non-overlapping windows, and a shared diffusion backbone and evaluation protocol across objective variants. Interpolation and suffix-missing forecasting are evaluated with specified masking, sampling, and seed procedures.

Appendix B. Implementation Details

  • Data are split chronologically 70%/10%/20%; training-subset min–max statistics are reused for validation and test before non-overlapping fixed-length windowing.ETTh1, ETTm1, Household, and Appliances use length and stride 96; Electricity uses 192.Source: Appendix B · p. 25
  • Core objective variants share a DiT-style Transformer conditioned through AdaLN-Zero, optimizer, masking protocol, and diffusion configuration, differing only in target and/or loss weighting.Checkpoint selection uses validation performance, and compared variants share data scale, target positions, and evaluation protocol within each condition.Source: Appendix B · pp. 25–26
  • Interpolation tests random-point, contiguous-block, and persistent-outage masking at 10%, 20%, 30%, and 50%; suffix-missing forecasting predicts the final 24, 48, or 72 positions using interpolation-trained checkpoints.Forecasting introduces neither a separate architecture nor a forecasting-specific training objective.Source: Appendix B · p. 26
  • Each condition uses 30 DDIM samples with 50 inference steps, takes the sample median, and clamps normalized predictions to [0, 1] before inverse transformation.Source: Appendix B · p. 26
  • Main experiments average three runs using seeds 42, 43, and 44, matched across objective variants to control initialization, DP noise, shuffling, timestep sampling, and mask sampling.The reported hardware and software environment is specified in Table B.9.Source: Appendix B · p. 26, Table B.9 · p. 26
  • Table B.11 specifies the core training configuration: all objective variants share the sampling scheme, clipping threshold, noise multiplier, and optimization-step count under (ε, δ) = (10, 10^-5).The backbone configuration is summarized in Table B.10.Source: Table B.11 · p. 27, Table B.10 · p. 27

Appendix C. Seed Variability

Appendix C reports seed-range details for the progressive objective ablation, while Appendix D describes representative and validation-tuned ε-prediction baseline comparisons and their evaluation protocols.

Appendix C. Seed Variability

  • Table C.12 reports MSE and MAE min–max ranges across three seeds for the progressive objective ablation under 50% missingness; main-text ablation results are seed averages.The appendix uses the same datasets, objective variants, and missingness patterns as Table 2.Source: Appendix C · pp. 27–28, Table C.12 · p. 28

Appendix D. Baseline Comparison Details

  • Representative baseline comparisons use ETTh1, ETTm1, and Electricity under 50% random-point missingness, including PatchTST, TimesNet, iTransformer, and the TimeDiT ε-prediction baseline.Results use matched DP-SGD configurations indexed by ε = 8 and ε = 10 and a common preprocessing, normalization, masking, and evaluation protocol.Source: Appendix D · pp. 28–29
  • The validation-tuned ε-prediction baseline sweeps clipping threshold and learning rate on ETTh1, ETTm1, and Appliances under the main matched configuration (ε, δ) = (10, 10^-5).Threshold, learning rate, and checkpoint selection use validation loss only, followed by the same evaluation protocol as the main comparison.Source: Appendix D · p. 29

Appendix E. Full Interpolation Results

Across all five datasets and three missingness patterns, timestep-aware dynamic weighting achieves the lowest MSE and MAE in every dataset–pattern combination; the static-weighting sweep motivates schedule-aware compression.

Appendix E. Full Interpolation Results

  • Timestep-aware dynamic weighting achieves the lowest MSE and MAE in all 15 dataset–pattern combinations under the main matched DP-SGD configuration.Replacing ε-prediction with v-prediction substantially improves interpolation utility; static weighting adds improvement in some settings, followed by further improvement from dynamic weighting.Source: Appendix E · p. 29, Table E.13 · p. 30

Appendix F. Static Loss Weighting versus Timestep-Aware Dynamic Weighting

  • Moderate static loss-weighting coefficients can improve validation error, but validation error and gradient norm do not vary monotonically together.Static weighting compresses all diffusion timesteps uniformly, motivating schedule-aware, timestep-dependent compression.Source: Appendix F · pp. 30–31, Figure F.3 · p. 31

Appendix G. Additional Proof Details

The additional analyses formalize how deterministic timestep weights affect clipping under a fixed DP-SGD threshold, test utility across matched privacy configurations, and isolate weighting effects in controlled gradient diagnostics.

Appendix G. Additional Proof Details

  • With a fixed clipping threshold and deterministic, private-data-independent timestep weights, dynamic weighting reduces the gradient bound more strongly at late low-SNR timesteps.Static weighting uses one compression factor across timesteps, whereas dynamic weighting sets λt = ¯αt.Source: Appendix G · pp. 31–32

Appendix H. Sensitivity to Matched DP-SGD Configurations

  • Across three matched DP-SGD configurations, v-prediction with dynamic weighting has lower average MAE than ε-prediction in all 20 dataset–severity conditions.The method ordering is preserved for ϵ ∈ {6, 8, 10}, although absolute performance is not monotonic in every condition.Source: Appendix H · p. 33, Table H.14 · p. 33

Appendix I. Controlled Gradient Diagnostic

  • A controlled diagnostic compares weighting rules at a fixed model state, complementing training-time measurements that do not isolate the immediate weighting effect.The controlled comparison uses the same v-prediction target, separating weighting from target reparameterization.Source: Appendix I · p. 34, Appendix I.1 · p. 34

Appendix I.1. Diagnostic Setup

  • The diagnostic matches the checkpoint, inputs, masks, diffusion noise, and timesteps, changes only the weighting rule, and performs no parameter update.Its aggregate-gradient norms do not define per-example clipping events or overall clipping fractions.Source: Appendix I.1 · p. 34

Appendix I.2. Mixed-Timestep Diagnostic

  • Dynamic weighting reduces median aggregate-gradient norms on all three datasets, while the 95th-percentile change is small under mixed-timestep sampling.Gradient cosine similarities are 0.774, 0.822, and 0.754 for ETTh1, ETTm1, and Appliances, respectively.Source: Appendix I.2 · p. 35, Table I.15 · p. 35

Appendix I.3. Timestep-Stratified Diagnostic

The timestep-stratified diagnostic finds that dynamic weighting has a stronger effect toward late low-SNR timesteps, though it does not reduce every gradient statistic in every interval.

Appendix I.3. Timestep-Stratified Diagnostic

  • The weighting effect is timestep-dependent: at t = 0–99, cosine similarity is 1.000 on all three datasets, and differences become more pronounced toward late low-SNR timesteps.At t = 900–999, cosine similarity is 0.882, 0.880, and 0.899 for ETTh1, ETTm1, and Appliances, respectively.Source: Appendix I.3 · pp. 35–36
  • On ETTh1 at t = 900–999, dynamic weighting reduces the median aggregate-gradient norm from 8.510 to 0.094.ETTm1 and Appliances show the same qualitative late-timestep pattern.Source: Appendix I.3 · p. 36, Table I.16 · p. 36
  • The 95th-percentile norm does not decrease in every interval; on ETTh1 at t = 800–899, it changes from 25.630 to 25.885.The measurements indicate timestep-selective weighting rather than uniform reduction of every gradient statistic.Source: Appendix I.3 · p. 36

Appendix I.4. Relation to Training-Time Gradient Diagnostics

Training-time and controlled diagnostics examine complementary aspects of gradient behavior, with the controlled analysis isolating weighting at a fixed model state.

Appendix I.4. Relation to Training-Time Gradient Diagnostics

  • Training-time diagnostics characterize gradients for individual private examples during actual DP-SGD, whereas the controlled diagnostic isolates the immediate weighting effect at a fixed model state.The controlled measurements concern aggregate gradients, not per-example clipping events.Source: Appendix I.4 · pp. 36–37
  • In the controlled results, dynamic weighting changes relative contributions to the pre-clipping aggregate gradient, with a more pronounced effect toward late low-SNR timesteps.These observations are consistent with the training-time diagnostics and the theoretical mechanism characterized in Section 3.3.Source: Appendix I.4 · p. 37
  • The authors describe the controlled and training-time observations as additional empirical support for the theoretical mechanism in Section 3.3.The two diagnostics address different parts of optimization: fixed-state weighting effects versus gradients encountered during DP-SGD training.Source: Appendix I.4 · p. 37
Loading 2610.00209v1…