Source-linked AI summary

Conditional Tensor Diffusion: Distributional Counterfactual Learning and Inference

Xinbing Kong, Zeyu Li, Junfan Mao, Bin Wu

arXiv:2609.25924v1stat.MLcs.LGecon.EM

TL;DR

In high-dimensional panel or tensor settings, intervention outcomes are observed where treatment occurs, but the corresponding control outcomes are missing, and point estimates do not capture their joint uncertainty. CFT-DIFF uses a treatment-masked conditional diffusion model with Tucker structure to learn the missing outcomes’ conditional distribution. It improves recovery in simulations and iFlex validation, while its distributional predictions distinguish pricing interventions by both demand reduction and target reliability.

  • Problem

    In high-dimensional panel or tensor settings, the joint conditional distribution of missing control outcomes is needed for uncertainty and joint-target questions that point estimates cannot answer.

    Source: §1 · pp. 1–2
  • Method

    CFT-DIFF conditions on observed control outcomes and the treatment mask while learning the nonlinear score in a low-dimensional Tucker core.

    Source: §1 · p. 2, §3.2 · p. 11
  • Results

    CFT-DIFF improves recovery over the considered causal panel and matrix/tensor completion methods, and iFlex validation confirms gains across three treatment patterns.

    Source: §1 · p. 3, §7 · p. 35
  • Takeaways & Limitations

    In the iFlex causal analysis, price signals with similar average demand reductions can differ in their probability of achieving a specified peak-demand reduction.

    Source: §1 · p. 3, §7 · p. 35

    The paper identifies pretreatment covariates, more complex treatment patterns, and heavy-tailed outcomes as directions for future extension.

    Source: §7 · p. 35

Abstract

from arXiv · show

Causal inference guides operational and managerial decisions but remains challenging in high-dimensional panel or tensor settings, where decisions may depend on the joint conditional distribution of missing control outcomes. We develop \emph{Counterfactual Tucker Diffusion} (\CFTDiff), which integrates the treatment mask and latent Tucker structure into conditional diffusion to recover this distribution given observed control outcomes through efficient nonlinear score learning in a low-dimensional core. The masked Tucker score preserves dependence across tensor modes while reducing the dimension of nonlinear score learning from the product of mode dimensions to the much smaller product of Tucker ranks. We establish high-probability error bounds for conditional score estimation that depend on the Tucker ranks, largest mode dimension, and the factor-strength-adjusted number of missing outcomes, and show how these bounds translate into recovery guaranties for the conditional distribution of the missing control outcomes. Across missing rates, simulations show more accurate point recovery than common causal panel and matrix/tensor completion methods; comparisons with nested diffusion specifications further demonstrate the gains from masked conditioning and Tucker dimension reduction. In Norway's iFlex experiment, \CFTDiff recovers missing outcomes more accurately than competing methods; when applied to causal analysis, its estimated conditional distributions yield counterfactual prediction intervals and target-attainment probabilities, allowing pricing interventions to be evaluated by demand-reduction magnitude and reliability.

1. Introduction

CFT-DIFF recovers the joint conditional distribution of missing control outcomes, addressing settings where point estimates cannot answer questions about uncertainty or joint target attainment. Its masked Tucker score combines observed controls with low-rank structure, and its theoretical and empirical results support distributional counterfactual analysis.

1. Introduction

  • The central problem is recovering missing control outcomes jointly because separate point estimates cannot quantify uncertainty or evaluate probabilities involving multiple outcomes.Tensor dimensions can grow as the product of mode dimensions, and treatment creates structured missingness that makes direct conditional-distribution learning demanding.Source: §1 · pp. 1–2

1.1. Contributions

  • CFT-DIFF learns a masked Tucker score that conditions on observed control outcomes while confining nonlinear score learning to a low-dimensional core.The Tucker core has r coordinates versus p for the full tensor, with r ≪ p.Source: §1 · p. 2
  • The conditional-score bounds depend on Tucker ranks, the largest mode dimension, and a factor-strength-adjusted missing-region size, and transfer to conditional-distribution recovery and nominal-coverage prediction intervals under stated conditions.The interval guarantee does not require a separate Gaussian approximation or asymptotic variance estimation.Source: §1 · p. 2
  • CFT-DIFF achieves the lowest point-recovery errors among the tested causal panel, matrix/tensor completion, and nested diffusion methods in iFlex validation, and improves distributional recovery over both nested specifications.The causal analysis also finds that price signals with similar average demand reductions can differ substantially in their probability of reaching a specified peak-demand reduction.Source: §1 · p. 3
  • Repeated draws from the estimated joint conditional distribution yield point estimates, quantiles, prediction intervals, and target-attainment probabilities from one model.The framework covers multi-way outcome arrays, including vector and matrix settings, and accommodates structured treatment patterns under stated conditions.Source: §1.1 · pp. 3–4

1.2. Related Literature

  • The paper targets the joint conditional distribution of missing control outcomes given observed controls, distinguishing it from methods focused on marginal counterfactual distributions or treatment-effect uncertainty.Prior causal panel and completion methods primarily target point counterfactuals, common components, or low-dimensional causal summaries.Source: §1.1 · p. 3, §1.2 · p. 5

2. Diffusion Models: Preliminaries

Diffusion models perturb data toward noise, then use a learned score to reverse the process. Conditional diffusion keeps conditioning information fixed, learns the conditional score through denoising, and uses it to generate conditional draws.

2. Diffusion Models: Preliminaries

  • Diffusion models use a forward noising process and a reverse-time process that transforms noise back into data using scores learned from data.The forward process provides analytically noised observations for learning; the reverse process provides generation.Source: §2 · p. 6, §2.1 · p. 7

2.1. Forward Diffusion and Reverse-Time Generation

  • In the forward process, a Gaussian perturbation increasingly dominates the data component, making the terminal distribution close to standard normal when α_T is sufficiently small.The perturbation is represented using a time-dependent scale and independent Gaussian noise.Source: §2.1 · p. 7

2.2. Conditional Scores, Denoising Score Matching, and Generation

  • Denoising score matching replaces the unknown conditional marginal score with the analytically available transition score without changing the population objective’s minimizer.The trained score is substituted into the reverse process to produce repeated draws from the learned conditional distribution.Source: §2.2 · pp. 8–9
  • Conditional diffusion perturbs the target while leaving conditioning information unchanged, and its reverse process uses the score of the conditional distribution.The conditional marginal score is the regression function of the known transition score on the noised target and conditioning information.Source: §2.2 · pp. 7–9

3. Learning the Missing Control-World Distribution

This section develops CFT-DIFF for learning the conditional distribution of control outcomes missing at treated coordinates, using a fixed treatment mask, conditional diffusion, and low-rank Tucker structure. It presents the masked score’s core representation, estimation and generation procedures, and score-learning guarantees, including how Tucker compression separates core-map learning from loading-space estimation.

3.1. Counterfactual Target and Training Sample

  • Given a fixed treatment mask, CFT-DIFF targets the full conditional distribution of control outcomes missing at treated coordinates, learning from complete control tensors masked synthetically.Observed control outcomes condition generation; observed treated outcomes are reserved for subsequent causal analysis.Source: §3.1 · pp. 10–11

3.2. Masked Conditional Diffusion in the Treated Region

  • The forward process perturbs only missing treated-region control outcomes while observed controls remain fixed; learning their conditional score enables reverse-time generation.The reverse-time process is determined by the conditional score, which is the central object to be learned.Source: §3.2 · p. 11

3.3. Low-Dimensional Tucker Structure of the Conditional Score

  • A low-Tucker-rank control-world model captures dependence across tensor modes and reduces nonlinear score learning to an r-dimensional core, where a sufficient statistic pools both outcome branches.The pooled statistic combines information from noised missing outcomes and fixed observed controls.Source: §3.3 · pp. 12–13

3.4. CFT-DIFF: Estimation and Generation

  • The masked Tucker score is trained with complete control tensors and a fixed mask, then reverse dynamics initialized at large T generate missing outcomes conditional on observed controls.Independent runs yield samples from the estimated conditional distribution of the missing outcomes.Source: §3.4 · p. 14

3.5. Intrinsic-Dimensional Guaranties for Conditional Score Learning

  • Score-learning guarantees are developed under light-tailed core concentration and Lipschitz regularity of the conditional core regression within a high-probability core region.These restrictions apply to the r-dimensional regression rather than the full score and are weaker than corresponding global smoothness conditions.Source: §3.5 · p. 15
  • Masking, precision weighting, and mode-wise transformations are represented exactly, so nonlinear approximation depends on r core coordinates and time rather than p tensor entries.The Core-Net width scales as ϵ^-(r+1), without an exponent involving the full dimension p.Source: §3.5 · pp. 15–16
  • The score approximation result fixes population loading spaces and idiosyncratic variances; its error sensitivity includes a factor h^-2 and motivates early stopping at t0 > 0.The subsequent generalization result allows the loading spaces and variance parameters to be learned from training samples.Source: §3.5 · p. 16
  • With probability at least 1 − 1/n, the learned conditional-score error bound separates nonlinear core-regression learning from mode-wise loading-space estimation.The core-regression term has leading rate n^-2/(r+5) up to logarithmic factors; the loading-space cost depends on pmax rather than vectorized dimension p.Source: §3.5 · p. 16
  • The score error is normalized by effective dimension d_T,β = r + p_T p^-β, where p_T counts missing control outcomes.With fixed mode ranks, d_T,β remains bounded in the strong-signal regime p_T ≲ p^β.Source: §3.5 · p. 16

4. Counterfactual Distribution Recovery

The recovery results connect conditional-score accuracy to recovery of missing-outcome distributions and weighted summaries, then use conditional draws to form counterfactual prediction intervals. Guarantees depend on score and loading-space estimation, early stopping, and the interval coverage condition.

4.1. A Score-to-Distribution Transfer Bound

  • The score-to-distribution transfer bound separates integrated score-estimation error from initialization error, which decays exponentially with diffusion horizon T.The comparison uses a common early-stopping time; the resulting early-stopping error vanishes as t_0 decreases to zero.Source: §4.1 · p. 17

4.2. Recovery of the Missing-Outcome Distribution

  • Theorem 4.1 gives a high-probability recovery guarantee for the generated conditional distribution; its error reflects nonlinear core learning, loading-space estimation, and variation in the treated region.When the scaled recovery error vanishes, average conditional KL divergence and total variation distance to the early-stopped target converge to zero.Source: §4.2 · p. 18

4.3. Recovery of Weighted Counterfactual Summaries

  • For weighted counterfactual summaries, distributional recovery is assessed on the fluctuation scale √n_e V_n_e,w,0 to avoid the mechanical contraction of an average.The stated consistency conditions include vanishing forward perturbation and conditional-distribution estimation error.Source: §4.3 · p. 20

4.4. Counterfactual Prediction Intervals

  • Conditional draws yield prediction intervals for unobserved counterfactual outcomes and, after translation by observed treated outcomes, for treatment-effect contrasts.The intervals use conditional quantiles directly, without a Gaussian approximation or asymptotic-variance estimation; coverage requires an anti-concentration condition.Source: §4.4 · pp. 20–21

5. Simulation Study

In simulations with known conditional distributions, CFT-DIFF improves point recovery over the established methods and nested diffusion benchmarks, while also improving distributional and interval recovery. The study varies missing-region size and factor strength, with the main text reporting the more challenging β = 0.50 setting.

5.1. Simulation Setting

  • The simulation evaluates recovery in missing regions at 25%, 50%, and 75% missing rates, comparing causal panel estimators, completion methods, and two nested diffusion benchmarks against exact conditional distributions.Each replication contains 360 complete training matrices and 12 independently generated test matrices.Source: §5.1 · p. 22, §5.2 · p. 23

5.2. Simulation Results

  • 17%–30% lower MAE than SC, and 15%–25% lower MAE than the next-best method even including nested diffusion benchmarks, across the reported designs.SC is the strongest of the five established point-recovery methods; Table 1 reports recovery across missing rates.Source: Table 1 · p. 23, §5.2 · p. 23
  • TUCKER-DIFF improves on CONV-DIFF, while CFT-DIFF adds a further improvement by conditioning on observed control outcomes through the treatment mask.The nested comparisons distinguish gains associated with Tucker dimension reduction and masked conditional construction.Source: §5.1 · p. 22, §5.2 · p. 23
  • Relative to TUCKER-DIFF, CFT-DIFF reduces Energy and CRPS by 14%–31%, with 90% coverage between 0.915 and 0.923 and intervals 18%–38% narrower.The reported IS and WIS reductions accompany coverage close to the nominal 0.90 level.Source: §5.2 · p. 24
  • At a 25% missing rate, CFT-DIFF histograms track the exact conditional densities’ location and concentration more closely than CONV-DIFF and TUCKER-DIFF.This pointwise comparison concerns entries near lower, central, and upper realized values and shows differences beyond generated-mean accuracy.Source: Figure 2 · p. 24, §5.2 · pp. 24–25

6. Distributional Demand Response to Dynamic Electricity Pricing

Using Norway’s iFlex experiment, this section evaluates recovery of missing no-policy demand and uses counterfactuals to estimate responses to dynamic pricing. It examines demand reductions during high-price hours, variation in target reliability across signals, and prediction intervals for average no-policy outcomes.

6.1. The iFlex Experiment and Counterfactual Structure

  • The analysis masks each complete intervention-day row so potentially treatment-affected hourly outcomes do not condition recovery of the no-policy counterfactual.The temporal exclusion restriction assumes no carryover to later non-intervention days and no anticipatory effects on earlier ones; reported reduced consumption on non-intervention days suggests this assumption may fail.Source: §6.1 · p. 27

6.2. Counterfactual Recovery Accuracy

  • At a missingness proportion near 0.284, CFT-DIFF achieved the lowest MAE and RMSE across all three patterns and remained stable as missingness increased.Under switchback, its MAE and RMSE were 0.0289 and 0.0449, versus 0.0299 and 0.0455 for the second-best method. TFI error increased substantially as the missing region expanded, especially under switchback and staggered adoption.Source: Figure 4 · p. 28, §6.2 · pp. 28–29, Table 2 · p. 29
  • CFT-DIFF improved distributional recovery, with lower energy score, CRPS, interval score, and WIS while retaining coverage close to the nominal 0.90.Its energy score was 0.1314 versus approximately 0.148 for both nested specifications, and CRPS was 0.0210 versus 0.0239. Coverage was 0.893 and average interval width was 1.2559, compared with 1.7246 for CONV-DIFF and 1.8352 for TUCKER-DIFF.Source: Table 2 · p. 29, §6.2 · p. 30

6.3. Causal Estimands

  • Causal estimands compare assigned price signals with no-policy outcomes, measuring both reductions during signal-specific peak hours and relative reduction over the full day.The pooled response distribution combines variation across realized intervention events with uncertainty in generated no-policy counterfactuals; the target-attainment probability is the fraction of pooled reductions reaching a specified target.Source: §6.3 · pp. 30–31

6.4. Demand Response to Dynamic Pricing

  • Seven signals—all P and P0 signals—had positive peak-period estimates, and peak-period reductions exceeded full-day reductions for 12 of 14 signals.The largest estimated peak reduction was under P15: 2.51%, or 0.163 kWh per assigned household–day over the two-hour peak period. Figure 5 identifies P030, with a 30 NOK/kWh peak over two afternoon hours, as the clearest reduction; demand often exceeded its counterfactual outside high-price hours, consistent with load shifting.Source: §6.4 · p. 31, Table 3 · p. 32, Figure 5 · p. 32
  • Demand generally falls relative to the no-policy counterfactual during high-price hours, but cumulative reductions turn negative over the full day for profiles P and P0.Responses vary by signal in magnitude and timing; the pattern is consistent with, but does not establish, electricity use shifting to other hours.Source: §6.4 · p. 33

6.5. Counterfactual Distributions, Intervals, and Response Reliability

  • P030 yields a positive peak reduction in 89% of event–draw outcomes and at least 3% in 35%; P15 performs better than P030 at 3% and 5% targets.P030 has the highest probability of attaining lower targets, while A and B signals are generally concentrated at lower reductions and P signals place more of their distributions above zero.Source: §6.5 · pp. 33–34, Figure 7 · p. 34
  • CFT-DIFF’s mean √ne-scaled prediction-interval width is 0.764, versus 1.355 for CONV-DIFF and 1.441 for TUCKER-DIFF; the latter intervals are 79.0% and 90.3% wider on average.CFT-DIFF has the narrowest interval for every price signal; its narrower intervals retain empirical coverage close to nominal in artificial-mask validation.Source: §6.5 · p. 34

7. Conclusion

CFT-DIFF extends counterfactual analysis to the joint conditional distribution of missing control outcomes, with score bounds tied to low-rank structure. Simulations and iFlex validation support improved recovery, while the conclusion identifies settings for future extensions.

  • Repeated draws from one learned distribution preserve dependence and yield point estimates, quantiles, counterfactual prediction intervals, and target-attainment probabilities.Source: §7 · p. 34
  • Score-estimation difficulty depends on Tucker ranks, the largest mode dimension, and the effective number of missing outcomes, rather than the full tensor dimension alone.Source: §7 · pp. 34–35
  • Table 4 reports central 90% intervals for average no-policy outcomes; width ratios above one mean CONV or TUCKER intervals are wider than CFT intervals.Widths are scaled by √ne, while interval endpoints remain on the original kWh scale.Source: Table 4 · p. 35, §7 · p. 35
  • Simulations and iFlex artificial-mask validation show improved recovery, while similar average demand reductions can correspond to different target-attainment probabilities.Source: §7 · p. 35
  • Future extensions include pretreatment covariates, more complex treatment patterns, heavy-tailed outcomes, and decision-focused intervention selection or optimization.Source: §7 · p. 35

Endnotes

The endnotes clarify the model’s conditioning and diffusion conventions, the scope of inference, and details of the iFlex data and treatment-signal setup.

  • The reverse-time process reproduces forward marginal distributions; it does not retrace an individual forward sample path.Source: Endnotes · p. 36
  • The conditional generator conditions on the specified treatment mask; the assignment sampling distribution need not be modeled, even under randomized assignment.Source: Endnotes · p. 36
  • The stated prediction interval is not a sampling confidence interval for a population average treatment effect; population inference requires assumptions linking analysis tensors to the target population.Source: Endnotes · p. 36
  • The iFlex analysis uses the full-scale second phase; the first phase ended early during COVID-19 and missed the intended winter experimental window.Source: Endnotes · p. 36
  • Each treatment group had four prespecified candidate price signals, so not every signal was available to every group.Source: Endnotes · p. 36

Technical Proofs and Additional Results

The appendix contains technical proofs and additional results, organized around notation, proofs, framework details, simulations, and empirical analyses.

EC.1. Summary of Notation

The notation appendix groups the paper’s principal and supplementary notation by its role in the analysis.

EC.2. Proofs

The proofs develop the masked tensor score analysis, establishing score identities and error bounds through low-dimensional reduction, treatment precision weighting, and truncation, statistical error, and approximation. They then use risk, entropy, coupling, and conditioning arguments to derive conditional-distribution recovery and downstream inference guarantees, including bounds for random intervals.

EC.2. Proofs

EC.2.1. Proof of Proposition 3.1

  • The proof derives the treated-region conditional score from a low-dimensional statistic that combines treated and observed-control information, with the core posterior mean supplying the factor-score term.The statistic has lower effective factor noise when observed controls carry nontrivial factor information.Source: EC.2.1. Proof of Proposition 3.1 · pp. 43–44

EC.2.2. Proof of Theorem 3.1

  • The proof approximates the Lipschitz core regression with a ReLU network, controls its L2 error using sub-Gaussian tails, then lifts that approximation to the treated-region score.The construction first approximates on a truncated core domain and extends the network by clipping.Source: EC.2.2. Proof of Theorem 3.1 · pp. 45–47
  • Theorem 3.1’s proof expresses masked-score discrepancy as a treatment-precision-weighted core-score residual and bounds the treatment precision operator norm by h^-1.Source: EC.2.2. Proof of Theorem 3.1 · p. 47

EC.2.3. Proof of Theorem 3.2

  • Theorem 3.2’s proof decomposes raw DSM loss into truncation, statistical, and approximation terms.The decomposition is used to bound the raw loss of the empirical minimizer.Source: EC.2.3. Proof of Theorem 3.2 · pp. 47, 49
  • The truncation event has failure probability at most (3n^2)^-1, and the complement’s second-moment contribution is bounded at order n^-2.Source: EC.2.3. Proof of Theorem 3.2 · p. 48
  • The entropy argument attributes complexity J to the core-network cover and pmax to finite-dimensional loading and variance covers, treating D and the mode ranks as fixed.The parameter covers are converted into an internal score-class cover whose selected centers remain admissible.Source: EC.2.3. Proof of Theorem 3.2 · pp. 54–55
  • Theorem 3.2’s proof separates statistical, truncation, and approximation errors, converts denoising score-matching loss to score risk, and concludes the stated high-probability bound.The final bound follows after balancing approximation and statistical errors; the proof gives probability at least 1 − 1/n.Source: EC.2.3. Proof of Theorem 3.2 · pp. 56–58

EC.2.4. Proof of Theorem 4.1

  • Theorem 4.1’s proof bounds reverse-process error using Girsanov and terminal relative-entropy contraction, then combines these bounds to prove (4.4).The terminal entropy is controlled through Ornstein–Uhlenbeck contraction after establishing finite conditional relative entropy.Source: EC.2.4. Proof of Theorem 4.1 · pp. 58, 60

EC.2.5. Proof of Corollaries 4.1 and 4.2

  • The corollary proofs transfer distributional error to aggregate-outcome distance and interval coverage, with a DKW bound controlling finite-sample learned quantiles.The arguments use relative-entropy contraction, Pinsker’s inequality, and coupling; they establish (4.7), the coverage result (4.9), and finite-B quantile control.Source: EC.2.5. Proof of Corollaries 4.1 and 4.2 · p. 61
  • Multiplying generated values and empirical endpoints by √ne leaves their error unchanged.Source: EC.2.5. Proof of Corollaries 4.1 and 4.2 · p. 62
  • Averaging the pointwise bounds over contexts and the finite generation sample adds CB^-1/2 to (4.9), proving (4.10).The argument first fixes the random interval endpoints conditional on F_B and applies the total-variation and coupling bounds after scaling them by √ne.Source: EC.2.5. Proof of Corollaries 4.1 and 4.2 · p. 62

EC.3. Framework Details

The appendix details CFT-DIFF’s matrix implementation, masked Tucker architecture, training objective, and conditional sampling procedure. It also specifies how the method is instantiated for simulations and iFlex.

EC.3. Framework Details

  • For iFlex, each household is a 74 × 24 day-by-hour matrix; complete control-household matrices train mask-specific models, and the same calendar masks are used for conditional generation.The household index identifies training or analysis samples, and the implementation uses order-two Tucker structure.Source: EC.3. Framework Details · p. 62, EC.3.4. Conditional Generation · p. 66

EC.3.1. Architecture

  • The score network separately encodes observed control outcomes and the noised missing region, processes their concatenated representations in a low-dimensional Tucker core, then decodes to predict missing-region noise.A missing-region feature shortcut is combined with the decoded core output; observed control outcomes stay fixed.Source: EC.3.1. Architecture · pp. 62–63

EC.3.2. Core-Net Function Class

  • The theoretical Core-Net class controls approximation and complexity through bounded ReLU maps with depth, width, parameter, and time-Lipschitz restrictions.Its nonlinear map takes the r-dimensional core statistic and diffusion time as inputs; the experiments use a time-conditioned U-Net in core space.Source: EC.3.2. Core-Net Function Class · p. 64

EC.3.3. Masked DDPM Training

  • Training synthetically masks complete tensors, noises only missing outcomes, keeps observed controls fixed, and minimizes noise-prediction loss over the missing region.The forward input samples a diffusion step and Gaussian noise for each minibatch.Source: EC.3.3. Masked DDPM Training · pp. 64–65

EC.3.4. Conditional Generation

  • Conditional DDIM generation starts from Gaussian noise in the missing region and updates only that region, preserving observed controls throughout each reverse trajectory.For iFlex recovery, 100 completions are generated per held-out household; their mean gives point estimates while the draw set supports distributional evaluation.Source: EC.3.4. Conditional Generation · p. 65

EC.4. Simulation Details and Additional Results

The appendix reports additional recovery results under stronger factors and gives the simulation and pointwise diagnostic designs. At β = 0.75, CFT-DIFF retains the best diffusion-method recovery scores across missing rates.

EC.4.1. Detailed Simulation Design

  • Simulations use 64 × 64 matrices with 16 × 16 latent cores, 372 generated matrices per replication, and missing rates of 25%, 50%, and 75%, evaluated over missing regions.Five replications use common training and evaluation matrices across methods and missing rates; β ∈ {0.50, 0.75} varies factor strength while preserving loading directions.Source: EC.4.1. Detailed Simulation Design · pp. 66–67
  • Pointwise diagnostics select entries near conditional percentiles 0.25, 0.50, and 0.75, while aggregate tables—not selected entries—provide the full-sample comparison.Selections balance CFT-DIFF discrepancy, improvement over benchmarks, and proximity to the target percentile.Source: EC.4.1. Detailed Simulation Design · p. 68

EC.4.2. Additional Recovery Results

  • At β = 0.75, CFT-DIFF has the lowest MAE, RMSE, Energy, CRPS, IS, and WIS among conditional generators at every missing rate; versus TUCKER-DIFF, Energy and CRPS fall 14%–42% and intervals narrow 17%–45%.Table EC.3 reports the comparison across three missing rates.Source: EC.4.2. Additional Recovery Results · p. 68, Table EC.3 · p. 69

EC.4.3. Additional Pointwise Distribution Diagnostics

  • At β = 0.50, CFT-DIFF’s pointwise discrepancies are 0.298–0.361 versus values above 1.17 for both diffusion benchmarks; CONV-DIFF is overdispersed, while TUCKER-DIFF retains excess dispersion, especially with eight factors.CFT-DIFF remains closer to the exact law at β = 0.75 even as conditional distributions become more concentrated.Source: EC.4.3. Additional Pointwise… · p. 69

EC.5. Additional Empirical Analysis

The section extends empirical analyses of CFT-DIFF recovery across missingness patterns, group sizes, and point, distributional, and interval metrics. It also examines demand-response variation by intervention design and spacing, outdoor temperature, region, and household characteristics.

EC.5.1. Overview of the iFlex Experiment

  • Across three sample constructions, day-mode spectra reached 95% energy with few components, while hour-mode spectra exceeded 95% in the first few and approached 99% well before 24, supporting a low-rank day-by-hour representation.The profiles were similar across globally nonintervention days, the pretreatment period, and control-group working days.Source: EC.5.1. Overview of the iFlex Experiment · p. 71, Figure EC.7 · p. 76

EC.5.3.1. Evaluation Measures

  • Evaluation uses MAE and RMSE for point recovery, energy score for joint 24-hour trajectories, and CRPS for marginal hourly distributions.Smaller values indicate better recovery for each of these measures.Source: EC.5.3.1. Evaluation Measures · pp. 73–75

EC.5.2. Missingness Patterns

  • The recovery tests mask complete household-day trajectories after the first intervention day under switchback, staggered-adoption, and simultaneous-adoption designs, while control households remain observed.The patterns cover both intermittent and monotone block missingness.Source: Figure EC.8 · p. 76

EC.5.3.1. Evaluation Measures

  • For missing-trajectory totals, coverage and interval width assess calibration and sharpness, while interval score and WIS summarize interval-based distributional recovery.The reported WIS aggregates the predictive median and central intervals at 50%, 60%, 70%, 80%, 90%, and 95% coverage.Source: EC.5.3.1. Evaluation Measures · pp. 76–77

EC.5.3.2. Results on Counterfactual Recovery Accuracy

  • CFT-DIFF yields the lowest RMSE across switchback, staggered-adoption, and simultaneous-adoption designs and remains stable as missingness expands, while alternative diffusion models deteriorate faster and TFI is especially sensitive.This shows the point-recovery advantage is not specific to the MAE criterion.Source: EC.5.3.2. Results on Counterfactual… · pp. 77–78, Figure EC.9 · p. 78
  • CFT-DIFF remains among the most accurate methods for household-day totals under all three missingness patterns and is more stable than alternative diffusion models as missingness increases.The analysis aggregates each missing 24-hour trajectory before evaluating recovery, showing that pointwise performance carries over to aggregate outcomes.Source: EC.5.3.2. Results on Counterfactual… · pp. 78–79, Figure EC.10 · p. 78
  • CFT-DIFF has the lowest CRPS and WIS at every reported missingness level, with its distributional advantage covering hourly marginals and household–day totals across all three patterns.Scores rise as the missing region expands, but more slowly for CFT-DIFF.Source: EC.5.3.2. Results on Counterfactual… · pp. 79–80, Figure EC.12 · p. 80
  • Under staggered adoption, the 90% interval width falls from 2.2037 for TUCKER-DIFF to 1.7198 for CFT-DIFF, with CFT-DIFF coverage of 0.920.At the missingness level closest to the application, CFT-DIFF has the lowest energy score, CRPS, interval score, and WIS under all three patterns; switchback and simultaneous adoption show the same lower scores and sharper intervals.Source: EC.5.3.2. Results on Counterfactual… · p. 80, Table EC.6 · p. 81

EC.5.3.3. Counterfactual Trajectory Recovery

  • Across nested household groups, CFT-DIFF tracks observed trajectories while retaining intraday variation; as groups grow, its median aligns more closely and intervals concentrate.Across random groups, it preserves more intraday variation, and results are not driven by a particular household composition; CONV-DIFF and TUCKER-DIFF often smooth peaks, while TFI can be unstable for small groups.Source: EC.5.3.3. Counterfactual Trajectory… · pp. 81–82
  • CFT-DIFF remains closer to observed trajectories for high-volatility and high-demand groups and across masked days, while other diffusion models often attenuate peaks or sharp movements.Counterfactual bands widen for more volatile groups; CFT-DIFF also follows changes in demand level, peak timing, and magnitude across days.Source: EC.5.3.3. Counterfactual Trajectory… · pp. 83–85

EC.5.4. Additional Results on Demand Response

  • Hourly demand-reduction distributions have medians generally close to zero but remain wide across household–policy-days, and expressing effects relatively does not materially change this pattern.Variation is especially visible during daytime and evening hours; some observations show sizable reductions, while others show little change or increases at particular hours.Source: EC.5.4. Additional Results on Demand… · p. 86
  • Peak-demand reductions increase with peak price most clearly for profile P, while estimates decline after the first exposure and peak for interventions spaced 7+ days apart.Most other profile comparisons are imprecise; repeated-exposure results provide limited evidence of systematic learning or fatigue. The temperature analysis estimates peak response across six temperature intervals and three price-profile classes, controlling for region, baseline demand, and intervention order.Source: EC.5.4. Additional Results on Demand… · pp. 87–88
  • Peak-demand reductions generally increase with outdoor temperature: estimates are strongly negative below −15°C, near zero around 0°C, and positive at higher temperatures.The increasing pattern appears across short, extended, and dual-peak price profiles, though some temperature–profile estimates are less precise; peak-period use is less responsive during very cold periods.Source: EC.5.4. Additional Results on Demand… · pp. 88–89
  • Regional differences persist after controlling for temperature, but the estimates indicate residual heterogeneity rather than causal effects of location.Relative reductions are positive in Oslo, Stavanger, and Trondheim, largest in Stavanger and Trondheim, while Bergen’s relative estimate is near zero and its absolute estimate is negative with a confidence interval excluding zero.Source: EC.5.4. Additional Results on Demand… · p. 89
  • Figure EC.34 shows nonmonotone variation across baseline-demand quintiles, load-volatility quintiles, and load-shape clusters, with wide, overlapping intervals for the cluster estimates.Q1 and Q4 baseline-demand groups show positive reductions, Q2 and Q5 negative responses, and Q3 is near zero; Q5 has the largest positive response among volatility groups, while C3 and C4 have the highest and lowest cluster point estimates.Source: EC.5.4. Additional Results on Demand… · pp. 90–91
  • Baseline-demand Q1 has the largest relative reduction (2.84%, 95% CI [1.72, 4.10]), while baseline-demand Q4 has the largest absolute reduction (0.581 kWh per household–policy day).Load volatility also shows a nonmonotone pattern: Q1 has a negative response and Q5 a positive response, with both confidence intervals excluding zero; intervals for peak-hour and evening-load shares include zero.Source: EC.5.4. Additional Results on Demand… · p. 92, Table EC.7 · p. 99
  • The intervention-spacing specifications group days since the previous intervention as 0, 1–6, or 7 or more and additionally control for intervention order.Estimates adjust for price signal, region, baseline demand, and temperature when available.Source: EC.5.4. Additional Results on Demand… · p. 100
  • Figure EC.32 reports pooled and price-profile-specific relative and absolute peak-demand reductions across outdoor-temperature bins.Pooled estimates adjust for price signal, region, baseline demand, and intervention order; profile-specific estimates adjust for region, baseline demand, and intervention order. Error bars are 95% intervals; positive values indicate demand reductions.Source: Figure EC.32 · p. 101, EC.5.4. Additional Results on Demand… · p. 101
  • Figure EC.33 reports relative and absolute peak-demand reductions by region.Estimates adjust for price signal, baseline demand, intervention order, and temperature when available; error bars are 95% intervals, and positive values indicate demand reductions.Source: Figure EC.33 · p. 101, EC.5.4. Additional Results on Demand… · p. 101
  • Figure EC.34 reports relative peak-demand reductions across household subgroups defined by baseline demand, load shares, load volatility, and load shape.Characteristics use pre-intervention electricity use; Q1–Q5 are quintiles and C1–C4 are clusters of normalized 24-hour load profiles. Error bars are household-bootstrap 95% intervals; positive values indicate demand reductions.Source: Figure EC.34 · p. 102, EC.5.4. Additional Results on Demand… · p. 102
Loading 2609.25924v1…