Adaptive Conformal Inference Under Delayed Feedback: Coverage Guarantees and a Delay-to-Memory Diagnostic

• With forecast horizon τ, a prediction's coverage is not observed until τ steps later, making adaptation depend on delayed feedback and raising the question of whether past residual information remains useful at the target.Source: §1 · p. 2
• The diagnostic r = τ/L compares forecast delay with the decay time scale of the signal driving non-exchangeability, and curve-collapse analysis tests it against raw delay τ.Source: §1 · p. 3
• Under AR(1), the delay-to-memory ratio strongly explains overall interval performance; under GARCH(1,1), it organizes raw interval scores more moderately but better identifies when scale normalization helps or hurts.Source: §1 · p. 3
• Under Markov switching, the ratio does not strongly collapse overall interval performance, but it characterizes how state-dependent normalization benefits change as delay grows relative to regime persistence.Source: §1 · pp. 3–4
• Across persistent processes, the ratio places delays from different time scales on a common axis, although its explanatory power depends on the form and location of temporal dependence.Source: §1 · p. 4
• The delayed ACI recursion decomposes into τ interleaved ACI-like sequences, yielding a finite-sample long-run coverage bound with explicit delay dependence and an approximate marginal bound tied to environmental change and γ.Source: §1 · p. 4

• The forecasting objective is to predict the target output at future time t + τ, with example horizons of 5, 10, 15, 20, 30, or 40 minutes.Source: §2 · p. 4
• At each time step, the forecasting model receives a history of recent sequential observations, represented by covariate-response pairs.Source: §2 · p. 4
• A fixed forecasting model μτ maps the recent history to a point prediction for the future response.Source: §2 · p. 4

• Online calibration evolves as observations arrive: each new outcome supplies a conformity score for assessing a past prediction, and the calibration set retains the most recent R scores.Source: §3 · pp. 4–5
• The quantile-based interval can be asymmetric; absolute residuals are a common alternative that yields a symmetric interval, and either score choice is compatible with the method.Source: §3 · p. 5
• The time-varying empirical quantile function uses available scores from predictions made at least τ steps earlier to construct the interval around the current forecast.Source: §3 · p. 5
• A large forecast horizon can leave the quantile function slow to reflect recent distribution changes, creating a tradeoff between horizon and calibration responsiveness.Source: §3 · p. 5

4 τ-Delayed Adaptive Conformal Inference

• τ-DACI adjusts the future miscoverage level from feedback arriving τ steps later, lowering α after undercoverage and raising it after overcoverage; τ = 1 recovers ACI.The update uses a fixed step-size γ.Source: §4 · p. 7

4.1 Alternative Form of τ-DACI

• The delayed recursion decomposes into τ interleaved phase sequences, each equivalent to a standard one-step ACI update when indexed by phase.Every time step belongs to exactly one phase, and the phase-wise form enables standard ACI analysis tools to be applied separately.Source: §4.1 · pp. 7–9

4.2 Boundedness of α(θ)

• Phase-wise miscoverage levels remain bounded by the standard ACI argument, adapted to use delayed quantiles and two-sided signed-residual prediction intervals.The bound prevents divergence into degenerate prediction sets.Source: §4.2 · pp. 9–10

4.3 Long-term Coverage Guarantees

• The long-run empirical-coverage bound is distribution-free beyond phase-wise boundedness; deviation decreases with T and is generally larger for longer horizons and depends on initialization.For τ = 1, the bound reduces to the result of Gibbs and Candès (2021).Source: §4.3 · p. 10

4.4 Approximate Marginal Coverage

• Because the long-run guarantee does not describe single-time marginal coverage, the section sets up its analysis under a stationary hidden Markov environment.The adaptive levels lie in [−γ, 1 + γ], and the initialization is drawn from the stationary distribution.Source: §4.4 · p. 10
• The result assumes Lipschitz continuity and a unique environment-specific optimal miscoverage level.Source: §4.4 · pp. 10–11
• The expected squared deviation of instantaneous marginal miscoverage from the target is bounded by a term for changes in optimal miscoverage across τ steps and a term for adaptation cost.The adaptation penalty grows linearly with γ.Source: §4.4 · p. 11
• When τ = 1, the bound reduces to the approximate marginal coverage bound of Gibbs and Candès (2021), while delay makes shift across τ steps explicit.Source: §4.4 · p. 11
• Choosing γ to minimize the bound’s right-hand side yields the stated optimal tradeoff between adaptation and stability.Source: §4.4 · p. 11

5 Evaluation Metrics

• Empirical coverage measures how closely observed coverage matches the nominal target 1 − α.A well-calibrated method should achieve empirical coverage close to 1 − α.Source: §5 · pp. 11–12
• Worst-case local coverage error (LCE_k) measures the largest deviation from nominal miscoverage across contiguous windows.It compares each window’s average miscoverage with α; the evaluation uses window lengths k ∈ {100, 250}.Source: §5 · p. 12
• The interval score (IS) evaluates calibration and sharpness together, with lower scores indicating narrower, well-calibrated intervals.It scores interval width when the observation is covered and adds a miss-magnitude penalty scaled by 2/α when it is not.Source: §5 · p. 12

6 Simulated Residual Processes and Diagnostic Framework

• Six controlled residual families isolate delay interactions; persistent AR(1), GARCH(1,1), and Markov-switching use τ-versus-r curve collapse, while mean and variance shifts use coverage-recovery duration.The ratio is r = τ/L, with L measuring how long past residual information remains informative; stronger collapse means interval scores align more closely across persistence configurations.Source: §6 · pp. 12–13

6.1 Experimental settings

• Simulations use T = 7000 steps, target miscoverage α = 0.1, and 10 seeds; γ⋆ is selected by interval score with a near-optimal coverage-aware rule.The rule first retains candidates within 1% of the minimum average interval score and favors empirical coverage of at least 0.89 when possible.Source: §6.1 · pp. 13–14

6.2 Gaussian — exchangeable baseline

• Independent Gaussian residuals with μ = 0 and σ = 1 provide the exchangeable baseline for evaluating adaptive methods.This represents a stable forecasting setting with independent, irreducible residual noise.Source: §6.2 · pp. 14–15

6.3 AR(1) — temporal dependence

• For stationary AR(1) residuals, memory length L = −1/log φ marks autocorrelation decay to e^-1, and r = τ/L compares delay with that time scale.The configurations vary φ from 0.1 to 0.99 and span r from approximately 0.01 to 92.Source: §6.3 · p. 15

6.4 GARCH(1,1) — volatility clustering

• GARCH(1,1) models volatility clustering: dependence persists in residual magnitudes, so conditional-scale normalization can stabilize residuals.Its volatility memory length tracks persistence in residual magnitude, unlike AR(1) memory, which tracks residual values.Source: §6.4 · p. 16
• The GARCH experiments compare unnormalized intervals with current-scale and target-scale oracle normalization to isolate delay-induced scale mismatch; both oracles are non-deployable diagnostics.The current-scale oracle uses σ_t for normalization and rescaling, while the target-scale oracle rescales using σ_(t+τ). Extended-delay diagnostics use α + β = 0.99 at τ = 100 and 300.Source: §6.4 · pp. 16–17

6.5 Mean and Variance Shift — Abrupt Changes in Residual Location and Scale

• Abrupt-shift simulations use three 800-step segments and measure the first coverage dip’s duration, with calibration windows R = 250 and R = 100 for combined shifts.Mean shifts vary Δ, while variance shifts vary the post-to-pre standard-deviation ratio; the shorter window removes pre-transition residuals within R steps.Source: §6.5 · p. 17

6.6 Markov-switching — stochastic and recurring regime changes

• The Markov-switching design models recurring regime changes with a two-state hidden chain and defines delay-to-memory using latent-state persistence, |λ2|^τ = e^-r_regime.It considers mean, variance, and joint switching; r_regime ≪ 1 indicates substantial dependence remains across the delay, whereas r_regime ≫ 1 indicates the chain has largely forgotten its state.Source: §6.6 · pp. 18–20

Appendix B.2 derives how latent-state persistence appears in the observed residuals and their

• Across switching families, regime memory is determined by the latent state process, while observed residual dependence can be attenuated or appear in scale rather than signed residuals.Symmetric transitions hold both regime frequencies and marginal residual distributions fixed across persistence levels; memory length, which governs autocorrelation decay, is distinct from mean dwell time.Source: Appendix B.2 · p. 20

Appendix B.2.

• Symmetric transitions give π0 = π1 = 0.5, allowing regime memory to vary without changing long-run regime frequencies or the marginal residual distribution.The appendix also reports dwell times across the persistence grid, distinguishing expected uninterrupted episode duration from memory length.Source: Appendix B.2 · p. 20
• The current-state and target-state oracles isolate delay-induced scale mismatch by differing in whether the target interval is rescaled to the target-time state scale; the experiment covers variance- and joint-switching regimes.Both oracles normalize by the current-state scale, while only the target-state oracle uses the target-time scale for interval rescaling.Source: Appendix B.2 · p. 20
• Interval scores are compared by whether the latent state changes across the forecast horizon, but mismatch status is only explanatory because intervals use a rolling calibration window.The mismatch indicator is Mt,τ := 1{At ≠ At+τ}.Source: Appendix B.2 · p. 21

7.1 Gaussian — exchangeable baseline

• Under exchangeable residuals, interval score worsens from approximately 4.15 at γ = 0.001 to roughly 4.43 at γ = 0.128, with a 7% efficiency loss and no coverage benefit across delays.Coverage failure occurs only at γ = 0.128, uniformly across τ, indicating that it is caused by the large step size rather than delay.Source: §7.1 · p. 21, Figure 1 · p. 22

7.2 AR(1) — temporal dependence

• Under AR(1), plotting interval score against rε reduces within-bin scatter by approximately 79%, with a transition near rε ≈ 1 and some persistence-dependent variation remaining.At rε ≲ 1, scores are generally below the Gaussian baseline; at rε ≳ 1, they approach or exceed it as relevant dependence decays.Source: §7.2 · pp. 21–22, Figure 2 · p. 23
• At φ = 0.99 and τ = 1, the AR(1) interval score is approximately 2.3 versus a Gaussian baseline near 4.15, while coverage falls to 0.83–0.86.At φ = 0.95, the advantage at small rε holds without a coverage cost; at φ = 0.99, part of the efficiency gain comes at the cost of coverage.Source: §7.2 · p. 22, Figure 2 · p. 23
• Under AR(1), the selected γ⋆ falls sharply as rε approaches one, then can rise above the grid floor when coverage-aware selection avoids undercoverage.When correlation survives the delay, larger steps can exploit structure; score-minimizing cells may undercover at higher persistence, while a coverage-compliant near-optimal step is typically available at little efficiency cost.Source: §7.2 · p. 23, Figure 3 · p. 24
• Under AR(1), the coverage-feasible region shrinks as persistence increases, and undercoverage can arise at large γ or, for persistent processes, small γ at large τ.Score-minimizing cells can mildly undercover, so selecting by interval score alone is insufficient; a coverage-compliant near-optimal step is typically available at little efficiency cost.Source: §7.2 · pp. 23–24, Figure 4 · p. 24

7.3 GARCH(1,1) — volatility clustering

• For α + β = 0.99, GARCH(1,1) interval score rises from approximately 3.86 at τ = 1 to approximately 4.59 at τ = 40; rε2 reduces scatter 46%, but no sharp threshold emerges.Substantial improvement over the i.i.d. Gaussian baseline occurs only for the most persistent process at the shortest delay.Source: §7.3 · p. 25, Figure 5 · p. 25
• For GARCH(1,1), using rε2 rather than τ reduces within-bin scatter in the current-scale oracle’s normalization benefit by approximately 50%, linking that benefit to volatility memory.Source: §7.3 · pp. 25–26
• For GARCH(1,1), current-scale normalization generally helps interval score when rε2 < 1, especially at higher persistence, but its advantage disappears or reverses as delay reaches or exceeds volatility memory.For α + β = 0.99, normalization becomes disadvantageous near rε2 = 1 and deteriorates further by rε2 ≈ 3.Source: Figure 6 · p. 26, §7.3 · p. 26
• The GARCH target-scale oracle’s score ratios stay below one and reach approximately 0.80 at rε2 ≈ 3 for α + β = 0.99, implicating prediction-time scale mismatch in longer-delay deterioration.The realized target-time scale is unavailable in practice, but more accurate forecasts of σt+τ could recover part of its benefit.Source: §7.3 · p. 26
• Across most GARCH configurations, current-scale normalization lowers worst-case local coverage error versus unnormalized intervals, including at delays where its interval-score advantage disappears.This suggests delay-induced scale mismatch affects interval efficiency more strongly than local coverage stability.Source: §7.3 · p. 27

7.4 Mean and variance shifts

• In shift experiments, adaptation duration is not monotonic in delay: larger delay-specific γ⋆ may strengthen corrections after feedback arrives even when initial coverage loss is deeper.For R = 250, duration fell from 131 to 114 steps at τ = 5 to 40 for ∆ = 4, and from 165 to 143 steps for r = 8.Source: §7.4 · p. 27
• For mean- and variance-shift trajectories, coverage failures deepen with shift magnitude and delay, but delay’s effect on failure duration is less systematic.Mean shifts cause sharp width expansions followed by gradual contraction; variance shifts cause sustained regime-specific width plateaus, with overcoverage as oversized intervals contract after return to baseline.Source: §7.4 · p. 27, Figure 7 · p. 28, Figure 8 · p. 28

7.5 Markov-switching — stochastic and recurring regime changes

• Replacing delay τ with rregime reduces within-bin interval-score scatter by 9%, 27%, and 10% for mean, variance, and joint switching; normalization scatter falls 83% and 84% for variance and joint switching.The normalization reductions are for the current-state oracle.Source: §7.5 · p. 30
• In Markov switching, p = 0.99 gives the lowest interval score at the shortest delay but deteriorates rapidly as rregime increases; absolute scores are not directly comparable across residual families.The Markov-switching designs were not standardized to unit unconditional variance, and their marginal residual dispersions differ.Source: §7.5 · pp. 30–31
• Under Markov switching, interval scores remain higher for p = 0.97 and p = 0.99, and the mismatch penalty is larger for more persistent chains.Mismatch-minus-match differences are close to zero for less persistent chains, remain positive at p = 0.97, and are largest at p = 0.99.Source: §7.5 · p. 31
• Current-state normalization becomes less effective as the regime delay-to-memory ratio increases, while the target-state oracle does not show the same deterioration.The target-state oracle uses future state information and is not deployable; its results benchmark potential gains from better target-time regime or scale forecasts. Normalization gains are smaller under joint switching because it does not address state-dependent means.Source: §7.5 · p. 31

7.6 Cross-family comparison of delay-to-memory scaling

• Replacing raw delay with the delay-to-memory ratio most strongly organizes interval performance for AR(1), with smaller gains for GARCH(1,1) and only weak-to-moderate organization for Markov switching.The contrast reflects where persistence appears: in residual level for AR(1), conditional variance for GARCH(1,1), and regime changes for Markov switching.Source: §7.6 · p. 33
• The ratio reduces scatter by 50% for the GARCH current-scale oracle and by more than 80% for the current-state oracle under variance and joint Markov switching.For Markov switching, the ratio organizes state-dependent scale information more clearly than overall interval performance.Source: §7.6 · p. 33, Table 7 · p. 33

8 Conclusion

• The τ-delayed recursion separates into τ interleaved sequences, yielding a finite-sample long-run coverage bound with explicit delay dependence.A complementary approximate marginal coverage bound relates deviation to environmental change across the forecast horizon and adaptation rate γ.Source: §8 · p. 34
• The paper’s simulations test whether the delay-to-memory ratio organizes performance across distinct forms of residual dependence.The conclusion’s supported framing is to interpret delay relative to the persistence of the residual signal, rather than raw time steps alone.Source: §8 · p. 34
• The delay-to-memory ratio is a diagnostic of whether prediction-time information remains relevant at the forecast target, not a universal performance measure.Its usefulness depends on which residual feature carries temporal dependence and which quantity is evaluated; memory may concern residual level, magnitude or variance, or a latent regime.Source: §8 · p. 34
• The appropriate adaptation rate γ depends on residual dynamics and the relation between delay and persistence.Under exchangeable residuals, aggressive adaptation reduces efficiency without improving coverage; with informative temporal structure or abrupt distributional change, faster adaptation can be useful.Source: §8 · p. 34
• The conclusion motivates aggregation across candidate learning rates and using residual persistence to guide how delayed feedback influences ACI updates.AgACI and FACI are cited as templates for automatic step-size adaptation.Source: §8 · p. 34

A Proofs for Coverage Guarantees Under Delayed Feedback

• The appendix collects derivations supporting boundedness and the long-run and approximate marginal coverage results for τ-delayed ACI.The boundedness argument uses a Robbins–Monro-style update and accounts for τ-step delayed, two-sided coverage feedback.Source: A Proofs for Coverage Guarantees Under… · p. 35, §A.1 · p. 35

A.1 Boundedness of α(θ)

• The boundedness argument shows that α(θ) cannot drift further outward under the phase-specific update.The update reflects delayed feedback and a two-sided coverage interval arising from the signed-residual construction.Source: §A.1 · p. 35

A.2 Long-Term Coverage Guarantees

• Partitioning time into τ phase sets gives phase-wise coverage bounds that combine into a global long-run coverage bound.Each phase set contains the indices updated by its sequence; the global calculation aggregates the phase-wise empirical miscoverage.Source: §A.2 · p. 36

A.3 Approximate Marginal Coverage

• Under stationarity, Lipschitz continuity, and a unique optimal level, the delayed marginal coverage bound separates horizon-related distribution shift from an adaptation penalty controlled by γ.The shift term is expressed through changes in the optimal level across τ steps.Source: §A.3 · pp. 37, 40

B.1 Autocorrelation of the two-state regime indicator

• The two-state latent regime indicator has autocorrelation exactly λ2^k, without an attenuation factor.For negative λ2, the autocorrelation alternates in sign while its magnitude decays as |λ2|^k.Source: §B.1 · p. 42

B.2 Autocorrelation of state-dependent residual features

• Conditioning on the two latent states makes the residual features conditionally independent, reducing their covariance to covariance between state-dependent conditional means.The derivation applies total covariance to the state pair at times t and t+k.Source: §B.2 · pp. 43–44
• Residual-feature autocorrelation inherits the regime process’s geometric decay factor, while the conditional-mean contribution determines its magnitude.cY is the fraction of feature variance attributable to differences in conditional means across states.Source: §B.2 · p. 44
• In mean switching, independent Gaussian noise attenuates residual autocorrelation but leaves its geometric decay factor unchanged.Source: §B.2 · p. 45
• Under variance switching, signed residuals are serially uncorrelated despite regime dependence, whereas absolute and squared residuals reveal it.Source: §B.2 · pp. 45–46
• With joint switching, state-dependent variance changes the autocorrelation attenuation coefficient but not the geometric decay factor.The section’s summary states that the attenuation coefficient affects magnitude, while the signed residual remains uncorrelated under variance switching.Source: §B.2 · p. 47

C.1 AR(1)

• Worst-case local coverage error generally rises with rε within each persistence level, but curves remain separated across φ, with stronger persistence associated with larger error.Thus, rε organizes local-coverage variation within a persistence level but does not account for its magnitude across levels.Source: §C.1 · p. 47, Figure 12 · p. 48

C.2 GARCH(1,1)

• Normalization reduces local coverage error in nearly all GARCH configurations, with the largest gains for more persistent processes and lower errors at k = 250 than k = 100.The exceptions are τ = 1 at α + β = 0.9 and α + β = 0.95; the local-coverage advantage remains even when the interval-score advantage disappears for rε2 ≳1.Source: §C.2 · p. 48, Figure 13 · p. 49

C.3 Mean and Variance Shift

• The shift experiments plot rolling coverage and interval width along combined mean- and variance-shift trajectories, and show selected step sizes and local coverage error across delays.They include calibration windows R = 250 and R = 100; step sizes are selected using the coverage-aware interval-score criterion.Source: Figure 14 · p. 49, Figure 15 · p. 50, Figure 16 · p. 51

C.4 Markov Switching

• The Markov-switching figures plot achieved interval score and mismatch-minus-match conditional interval score, plus normalized-to-unnormalized interval-score ratios, against raw delay τ.Source: Figure 20 · p. 54, Figure 21 · p. 54