PaperlayerView exact revision on arXiv

Source-linked AI summary

The Premise Is the Problem: Exchangeability Failure in Self-Monitored Test-Time Adaptation

Weijia Han, Lisha Qu, Zhenda Li, Liying Liang

arXiv:2610.07038v1cs.LGstat.ML

TL;DR

Self-monitored adaptation uses a forecaster’s own feedback to update and monitor it, but the guarantee for its monitor depends on exchangeable scores. The paper analyzes overlapping labels and dependent forecast errors, finding that reuse can break this premise and produce crossings, while adaptation and protective responses also have costs.

  • Problem

    The paper asks whether an anytime-valid monitor’s exchangeability guarantee survives when the same prediction scores both drive adaptation and monitor the learner.

    Source: §1 · p. 1

  • Method

    The paper studies a block Kalman adapter and conformal test-martingale gate across feedback protocols, and audits score dependence with real-stream replays and a conditional null.

    Source: §3 · p. 3, §4.3 · p. 5

  • Results

    On the conditional null, label reuse raised classical no-reset wealth crossings from 2 or 3 of 60 to 58 or 59 of 60.

    Source: §4.3 · p. 5

  • Takeaways & Limitations

    A monitor is not a plug-in safeguard: validate its statistical assumptions and triggered actions in the full adaptive system.

    Source: §1 · p. 2

    The deployed gate’s sliding calibration window, wealth reset, and learned covariance depart from the classical construction, so it has no proven Ville bound of its own.

    Source: §3 · p. 3

Abstract

from arXiv · show

Modern forecasting models are often updated after deployment so they can respond to changing data. These updates can also make predictions worse, so practical systems need a reliable monitor that can detect harmful changes and trigger protection. A natural design is to monitor the same prediction errors that guide the updates. This paper asks whether the statistical guarantee behind such a monitor remains valid when monitoring and adaptation use the same feedback. We study this question in multi-step time-series forecasting. We show that overlapping targets and dependence in forecast errors can break a key assumption required by the guarantee. The monitor may then raise alarms even when no harmful change has occurred, and its response can further damage prediction quality. We also find that adaptation can hide sustained changes from its own monitor, while the original frozen model retains a clearer signal. These results expose a basic failure mode in self-monitored adaptation. They show why reliable deployment requires checking the monitor's assumptions, comparing adaptation with the frozen model under realistic feedback, and limiting the effect of every protective response.

1 Introduction

Self-monitored adaptation uses the same prediction scores to update a forecaster and monitor it, but the monitor’s anytime-valid guarantee requires exchangeable scores. The paper shows how overlapping labels can break that premise and reports consequences for adaptation and protective responses.

Motivation

  • Test-time updates can harm forecasts, so deployed adaptation needs a monitor that can trigger protective action.The proposed self-monitored design uses the adapter’s own prediction scores both to guide updates and to monitor them.Source: §1 · p. 1

Contributions

  • A false alarm can trigger a response that damages the adapter, and tested responses can also act too often on clean data or fail under sustained contamination.Removing reset and re-entry helps only the selected cohort.Source: §1 · pp. 1–2
  • Overlapping forecast labels can make correctly specified scores non-exchangeable, invalidating the classical guarantee; on the conditional null, reuse raises crossings from 2 or 3 of 60 to 58 or 59 of 60.Spacing origins one horizon apart restores validity for the truth-held scorer.Source: §1 · p. 1, Figure 1 · p. 2
  • On real streams, frozen-backbone residuals cross without a learner, while the adapter absorbs sustained change within one horizon and the frozen residual remains elevated.Under two deployable feedback protocols, the adapter loses to the frozen backbone in 29 of 30 backbone–stream evaluations.Source: §1 · pp. 1–2
  • The conditional-null finding is theoretical, while findings on real streams, learned scores, adapter value, and response design are empirical.The authors conclude that both the monitor’s assumptions and the actions it triggers need validation in the full adaptive system.Source: §1 · p. 2

2 Related work

Prior work provides anytime-valid monitors, deployed-model monitoring, dependence-aware conformal methods, and test-time adaptation. This paper distinguishes its question: whether exchangeability holds for scores produced by a learner that also adapts from them.

2.1 Anytime-valid testing and conformal martingales

  • Conformal martingales test exchangeability online, with a Ville bound on threshold crossings when scores are exchangeable; this paper asks whether that premise holds when the monitored learner produces the scores.The review distinguishes this exchangeability-based guarantee from other e-processes that target different nulls.Source: §2.1 · p. 2

2.2 Monitoring deployed models under shift

  • Prior monitors track deployed-model risk, secondary error models, or covariate shift, whereas this work studies score dependence inherited from forecast residuals and amplified by overlap and adaptation.Related work also covers heuristic drift detectors and gating adaptation on a detector.Source: §2.2 · p. 2

2.3 Conformal prediction under dependence

  • Dependence-aware conformal methods repair forecast intervals, while this paper examines exchangeability of the monitor’s scores and the consequence of overlapping observations for a monitor sharing scores with the learner.Proposition 1 restates overlapping-observation autocorrelation in forecast evaluation [Hansen and Hodrick, 1980, Diebold and Mariano, 1995] and adds the implied non-exchangeability.Source: §2.3 · p. 2

2.4 Test-time adaptation of forecasters

  • Test-time adaptation work includes methods for online forecasters and reset mechanisms for error accumulation, while this paper evaluates a dynamic linear model under immediate and leakage-free reveal protocols.The tested adapter is the dynamic linear model of West and Harrison [1997]; related work also includes a betting-martingale-driven adaptation design [Bar et al., 2024].Source: §2.4 · p. 3

3 The online learner and the gate under test

The paper pairs a block Kalman adapter for frozen 96-step forecasts with a conformal test-martingale gate, then tests them under several feedback protocols and a defined multi-stream evaluation frame.

The adapter

  • A block-diagonal Kalman filter calibrates a frozen backbone’s 96-step forecast with a slope and intercept per horizon, initialized to reproduce the frozen forecast.Online hypergradients adapt the forgetting factor and process- and observation-noise parameters after non-skipped feedback packets.Source: §3 · p. 3

The gate

  • The gate uses randomized conformal p-values and a fixed 19-point mixture of power martingales, but its sliding calibration window, wealth reset, and learned covariance have no proven Ville bound.A fire opens a recovery window; the spike check clips or skips extreme innovations only when no fire or recovery window is active.Source: §3 · p. 3

Feedback protocols

  • Feedback ranges from immediate and delayed reveal to composite-coordinate and stride-96 protocols, which alter which packets supply gate scores and adapter updates.A separate detector-stride-96 variant updates from every packet but sends only every 96th score to the wealth process; shadow variants save scores while the gate takes no action.Source: §3 · p. 3

Evaluation frame and terms

  • The evaluation covers 15 backbone–stream cells across three backbones and five streams, with three algorithmic replicates per cell and defined contamination and divergence rules.Panel summaries are descriptive and condition ratios on both runs surviving the fixed divergence guard.Source: §3 · p. 4

Experimental configurations

  • The primary configurations include ungated, Huber-only, full-gate, and fixed-cutoff Kalman variants, plus two SGD baselines and exploratory response ablations.The fixed-construction null tests the gate without a learner; the no-parameter-drift simulation runs the complete pipeline on a fixed-design regression stream.Source: §3 · p. 4

4 Premise failure on the deployed score streams

Clean-stream alarms persist across the deployed variants, and analyses trace them to score dependence rather than requiring parameter drift or adaptation. Overlapping labels break exchangeability in the controlled null, while disjoint origins restore validity only for the truth-held scorer.

4 Premise failure on the deployed score streams

  • All 135 clean runs of the three martingale-carrying variants fire at least once, and delayed reveal does not remove the pattern.The fixed-construction null fires in at most 1 of 60 runs, while the no-parameter-drift simulation fires in all 60.Source: §4 · p. 4

4.1 Element-level isolation

  • 11 variants that learn R online fire or cross in 60 of 60 runs, while eight that hold R fixed do so in 1 to 3 of 60.The element-level interventions attribute the excess crossings to the convergence transient in the learned observation-noise estimate.Source: §4.1 · p. 4
  • With R held fixed, its mean coordinate value is 0.0079 versus a true value of 0.0025 and the mean score is about 30; learning R makes the score rise from 90.2 to 96.1.Neither longer-warm-up variant fires in 60 runs; starting R at its true value reduces firing to 2 of 60, although Ljung-Box rejects in 58 of 60.Source: §4.1 · p. 4

4.2 The frozen-scale variant on the real streams

  • The frozen-scale variant fires in 45 of 45 clean runs under each reveal protocol, so the convergence transient does not explain its real-stream firing.It holds R at its warm-up value while λ and Q continue learning.Source: §4.2 · p. 4

4.3 The classical no-reset wealth process and origin overlap on real streams

  • The no-reset wealth process crosses in 180 of 180 real-score replays, while frozen-backbone residuals also cross without a learner, including 45 of 45 runs with horizon-specific fixed scales.Horizon-separated scoring reduces but does not eliminate crossings; the dependence precedes adaptation on four of five streams.Source: §4.3 · p. 5
  • Reusing overlapping labels raises no-reset crossings from 2 or 3 of 60 to 58 or 59 of 60 with the scorer at truth; spacing origins 96 apart restores validity only for that scorer.With the adapter learning, reuse raises crossings from 1 or 2 of 60 to 54 or 55 of 60, and thinning reduces observed crossings without a theoretical guarantee.Source: §4.3 · p. 5

5 Attribution of the damage to the soft reset and the protocol

On the immediate-reveal weather transient, the full gate can turn a brief event into divergence, while removing its soft reset avoids most measured damage. Delayed-reveal results do not isolate one component as the unique cause.

5 Attribution of the damage to the soft reset and the protocol

  • On the weather transient, the full gate produces divergence on two of three backbones on all three seeds; TimesFM 2.5 survives, finishing at 2.57–3.06 versus 0.36 for the ungated filter.The transient occurs at steps 18,536 to 18,538 on every backbone and seed under immediate reveal.Source: §5 · p. 5
  • The soft-reset variant alone reproduces the immediate-reveal divergence on all three seeds for TiRex and Chronos-2, and finishes above the ungated filter on every backbone across 20 clean-weather seeds.It also finishes above the full gate in 18, 14, and 13 of 20 paired seeds across the three backbones.Source: §5 · p. 6
  • Removing the soft reset while retaining recovery avoids most measured damage: the immediate-reveal clean-weather cell finishes at 0.136 to 0.434, and delayed results do not identify a unique causal component.The reset-free variant diverges in 1 of 450 immediate runs and 3 of 450 delayed runs, versus 60 and 2 for the full gate.Source: §5 · p. 6

6 Alternative monitors and adapter value

The adapter’s value depends on feedback protocol and stream: it often loses to the frozen backbone under deployable feedback, while frozen residuals can provide a stronger sustained-change signal. Tested monitors and protective responses also show false alarms or fail under some sustained shifts.

6 Alternative monitors and adapter value

  • Under composite-coordinate and delayed reveal, the adapter loses to the frozen backbone on 14 and 15 of 15 cells, with median gains of −0.34 and −0.60.Under diagnostic immediate reveal it gains on 12 of 15 cells but multiplies frozen error 2.9 to 22 times on weather; the ORCA refiner gains on all 12 non-weather cells, but differing inputs and warm-up rules prevent fair ranking.Source: §6 · p. 6
  • Under causal reveal, post-change adapter error is 0.16 to 0.73 of frozen error off weather but 2.0 to 323 times it on weather.Causal residual summaries recover at most 23 percent of frozen-model error, while an offline oracle using the adjacent label recovers 84 to 95 percent on four streams.Source: §6 · p. 6
  • In a selected cohort of 36 immediate-reveal runs with positive clean gain, the capped policy retains 0.96 of that gain under packet spikes versus −0.35 for the ungated filter; this does not establish general effectiveness.The full gate, fixed cutoff, and capped policy all lose under packet-level shifts and freezes.Source: §6 · p. 7
  • The fixed cutoff’s median clean action rate is 34.0 percent against a nominal 0.1 percent; its survivor-only ratio stays within ±0.05, yet two delayed-weather pairs diverge.Under the fixed 100-fold guard, adverse delayed outcomes occur in 40/45 full-gate runs and 18/45 cutoff runs.Source: §6 · p. 7, Figure 5 · p. 7
  • Within 960 steps at magnitude 1.0, the A = 400 e-detector alarms on frozen residuals in 28/45 runs, versus 0/45 clean twins and 0/45 adapter-residual runs.Frozen scores remain 14–17 times baseline through step 2,000; the hold-and-handover response trades contamination robustness against genuine-change adaptation.Source: §6 · p. 7
  • At 5 percent contamination, packet freezes degrade the fixed cutoff on four non-weather streams, while packet shifts degrade every tested ETTh2 variant and all Kalman variants on ETTm1 and traffic.The spike check therefore does not prevent sustained-change damage.Source: §6 · p. 7

7 Implications and scope

The paper distinguishes its theoretical result from empirical findings and limits the evidence to a specific set of models, streams, and forecast horizon. It also documents AI assistance, reproducibility materials, ethics, and the status of appendix analyses.

7 Implications and scope

  • The conditional-null result is theoretical; findings on real streams, learned scores, adapter value, and response design are empirical, with evidence covering one adapter, three backbones, five fixed paths, and horizon 96.Most cells use three seeds; the delayed-reveal clean-weather ablation uses 20 per backbone.Source: §7 · p. 8

AI use statement

  • Generative AI assisted interpretation of saved experimental outputs, drafting the Corollary 1 statement and proof, and manuscript editing; the authors checked interpretations and verified the proof line by line.The authors state AI was not used to generate synthetic data, form hypotheses, design methodology, implement code, or clean data.Source: AI use statement · p. 8

Reproducibility statement

  • The paper reports completed-cache reruns, saved records and action logs, supplementary code for experiment families, and plans to release the full codebase, records, logs, and forecast cache upon publication.External-refiner evaluation is a third-party diagnostic using released methods at default settings.Source: Reproducibility statement · p. 8

Ethics statement

  • The study uses five public forecasting benchmarks and three publicly released frozen models, involves no human subjects or personal data, and reports no conflicts of interest.The authors frame the subject as a deployed safety mechanism whose guarantee does not hold under the studied conditions.Source: Ethics statement · p. 8

Appendix roadmap.

  • The appendix roadmap classifies analyses by status rather than assigning a status to whole sections, and calls a follow-up decision-ruled only when its rule was fixed before the campaign.It distinguishes original outcomes, theory-guided premise analyses, recorded-rule follow-ups, exploratory follow-ups, and methods or audit material.Source: Appendix roadmap · p. 12

A Experimental specifications

The experiments pair a block Kalman adapter with several gates and feedback protocols, then test clean and contaminated streams alongside synthetic nulls. The specifications vary feedback, adaptation, and protective responses to isolate how the loop behaves.

A Experimental specifications

  • The synthetic nulls separate stream-level dependence from learner effects, while the experimental specifications use chronological splits and normalized target-scale forecasts and labels.The block-aware filter plus hyperparameter step costs 12,230 scalar operations per step, or 0.187× one specified SGD step, though its measured Python loop is slower.Source: A Experimental specifications · pp. 12–15, Table 5 · p. 15
  • The block Kalman adapter keeps a slope and intercept per forecast horizon and updates its forgetting factor and noise parameters from each eligible packet’s innovation likelihood.The λ update uses a diagonal approximation that drops within-horizon slope–intercept covariance; process- and observation-noise coordinates adapt separately.Source: A Experimental specifications · pp. 12, 14
  • A gate fire resets covariance, boosts process noise, and holds the effective forgetting factor at 0.95 for a 50-step recovery window; the capped response removes soft reset and re-entry.The deployed gate keeps its calibration window across fires and resets only wealth; re-entry means a fire during recovery restarts the window.Source: A Experimental specifications · pp. 12–13, Table 2 · p. 13
  • Contamination is injected into already-formed origin–horizon label packets at 1%, 5%, or 10% of test steps, using spike, level-shift, or freeze interventions.Spikes alter isolated entries by six robust scale units, level shifts apply a three-unit contiguous offset, and freezes hold a contiguous packet block at its onset value.Source: A Experimental specifications · p. 12
  • The main variants run under immediate reveal, while fifteen also run under delayed reveal; the frozen-backbone reference reads no labels and is unaffected by label contamination.The variants include Kalman and SGD baselines, with additional response and stride configurations in targeted analyses.Source: A Experimental specifications · pp. 12–13

B Reproducibility

The authors rerun experiments using a completed, immutable forecast cache and retain earlier incomplete-cache records only as provenance. The reruns largely reproduce prior ratios and preserve the stated verdict that the gate fails its preregistered comparison rule.

B Reproducibility

  • Possible pretraining overlap may affect forecast quality and limits broader generalization, although the validity analysis conditions on realized forecasts and scores without requiring a pretraining-exposure assumption.The five public streams’ exposure in released time-series checkpoints is difficult to establish.Source: B Reproducibility · p. 15
  • Each contaminated run is paired with a clean twin using the same cached forecast inputs, while three seeds are Monte Carlo replicates conditional on each fixed cached stream.The divergence guard truncates runs at a prespecified threshold, so saved records cannot support a higher-threshold sensitivity analysis without rerunning affected cells.Source: B Reproducibility · pp. 15–16
  • Of 300 completed-cache ratios, 263 move by at most 0.01 and 280 by at most 0.05; divergence agrees in 297 cells, and the full gate’s weather divergence recurs.The largest changes are itemized separately, and the main spike, freeze, and shift evaluations are checked on the completed cache.Source: B Reproducibility · p. 15
  • The composite gate rule fails on all three backbones, requiring the gated variant to beat the ungated variant on at least three comparable stream cells per contamination kind.Cells with less than 5% ungated degradation are excluded, and every evaluated contamination kind must pass.Source: B Reproducibility · p. 16

C Clean-stream fire census

Martingale-carrying variants fire repeatedly on clean streams, and delayed reveal does not eliminate the pattern. Diagnostics and intervention analyses further show that dependence, aggressive spike classification, and recovery behavior qualify how these fires should be interpreted.

C Clean-stream fire census

  • Across the three primary variants, clean-run fire counts vary widely by stream, reaching 459–557 for Huber-only on ETTm1 and 115–277 for the full gate on weather.Weather counts for the full gate are truncated by clean-stream divergence; raw counts are not rates because test-block lengths differ.Source: C Clean-stream fire census · p. 16
  • On clean streams, the fixed-cutoff variant classifies 28.6–41.4% of steps as spikes, far above the nominal 0.1% expected at the 0.999 chi-square quantile.Even at the 0.99999 quantile, its median classified share is 23.5%, or 23,500 times nominal.Source: C Clean-stream fire census · p. 16
  • Recovery-period overrides suppress 981–1,274 spike classifications per weather run, while applying every classification during recovery produces more weather divergences than the deployed gate.The comparison with the cutoff also depends on action budget and exposure; weather is not budget-matched in both directions.Source: C Clean-stream fire census · p. 17, Table 12 · p. 20
  • First-step innovation diagnostics reject uncorrelated residuals in 119 of 135 clean runs, but weather’s near-one p-values reflect lower-tail degeneracy rather than evidence of whiteness.An extreme innovation can drive sample autocorrelations toward zero even while those weather runs continue to fire repeatedly.Source: C Clean-stream fire census · p. 18
  • Delayed reveal does not reduce the full gate’s per-step clean-fire rate on any backbone, while consecutive origins still overlap in 95 of 96 entries.The delayed protocol’s first 96 test steps are predict-only, and its inflated-covariance start-up state can bring first fires earlier on some streams.Source: C Clean-stream fire census · p. 19
  • Every martingale-carrying variant fires more than 1,000 times per backbone across tested clean-stream protocols, except the stride-96 gate, which fires 68 times over 90 runs.The fixed-cutoff variant has no martingale and therefore has no fire event by construction.Source: Figure 6 · p. 21

D Classical no-reset replay and origin-overlap analyses

This section examines threshold crossings in classical no-reset monitoring on clean real-stream runs and frozen residuals, including after changes to scoring scale and causal refinement. It analyzes how overlapping labels, dependent score streams, label reuse, score thinning, clustering, and aggregation relate to crossing behavior and wealth.

D Classical no-reset replay and origin-overlap analyses

  • The classical no-reset process crossed in 45 of 45 runs in every clean-stream cell, versus 2.25 expected crossings per 45-run cell if scores were exchangeable.The observed streams are not assumed to satisfy that premise; the deployed gate also fired on the same streams, with first-fire medians of 515 and 138 steps under immediate and delayed reveal with the scale learned.Source: D Classical no-reset replay and… · p. 22, Table 14 · p. 23
  • Across 90 clean runs, the deployed gate fired in 90 of 90 at calibration lengths 100, 250, and 500 and levels down to 10^-6, while at 10^-8 it fired in 87 of 90 at length 500.The no-reset process crossed in 80 of 90 at 10^-8; fixed bets of 0.9, 0.5, and 0.1 crossed in 90, 43, and 5 runs, respectively, and nominal marginal-uniformity and serial-dependence diagnostics rejected every deployed and no-reset p-stream.Source: D Classical no-reset replay and… · p. 22
  • The SGD shadow stream crossed in 45 of 45 runs under each protocol, but its stride-96 live gate crossed in only 12 immediate-reveal and 10 delayed-reveal runs.The stride-96 feedback protocol reduced clean-stream firing from 45 of 45 runs with a median 41 fires to 7 of 36 non-diverged runs with one fire each; 3 of 15 cells hit the divergence guard, and the martingale saw 96 times fewer scores.Source: D Classical no-reset replay and… · pp. 22–23, Table 15 · p. 23
  • At stride 96, bursts of 16 origins crossed more often than single-origin samples in all 15 backbone-stream pairs, by at least 0.41, while real-score autocorrelation fell from 0.95 at lag 1 to 0.12 at lag 72.For the learned-scale shadow stream, pooled autocorrelation reached zero at lag 89, with residual positive floors on weather, traffic, and electricity; the burst analysis is observational because selected origins also change.Source: D Classical no-reset replay and… · pp. 23–24
  • In the reuse intervention, fresh-label oracle scores were i.i.d., whereas reused labels made adjacent-origin targets identical and the reused p-stream rejected serial-independence diagnostics in 60 of 60 runs.Under reuse, the lag-one autocorrelation was 0.98; the moving-sum analysis also shows that overlapping sums are dependent and therefore nonexchangeable, while spacing origins by the horizon yields i.i.d. scores under the proposition’s assumptions.Source: D Classical no-reset replay and… · pp. 25–26
  • Under heteroscedastic noise, the fixed-scale oracle crossed in 60 of 60 independent-label runs, while the learning adapter crossed in 21 of 60 across all tie-break streams.Heavy-tailed Student-t noise raised independent-label learning crossings to 9 of 60 across all streams; under Gaussian noise, the learning adapter’s low crossing frequency is only a measurement, not a consequence of the thinning corollary.Source: D Classical no-reset replay and… · p. 26
  • Permuting frozen residuals within blocks on real streams produced a median adapter MSE ratio of 1.06, with weather under delayed reveal reaching ratios of 18 and 21 at block sizes 192 and 512.Crossings varied substantially by stream and block size; 14 of 360 permuted runs ended at the divergence guard, and the intervention changes residual alignment as well as reuse.Source: D Classical no-reset replay and… · pp. 26–27
  • Single-horizon no-reset streams crossed in 6,319 of 8,640 run-horizon pairs, while a one-term diagonal stream crossed in 90 of 90 runs and a random-horizon stream in 181 of 270.The diagonal stream’s median peak log-wealth was 98.0 versus 48.9 for pooled scores; antidiagonal crossings fell to 58 of 90, indicating that reuse concentration—not simply the horizon cycle—drives the diagonal’s wealth, while thinning one-term scores reduced crossings substantially.Source: D Classical no-reset replay and… · p. 27

Origin-overlap analyses

  • Grouping the 96 terms by target timestamp lowered crossings to 75/90 and median peak wealth to 12.4, versus 90/90 and 48.9 for the pooled score.The one-term wealth was 4.9 without shared labels, 12.4 with aggregation but no shared label, and 98.0 with a shared label in the one-term statistic; the grid ranks the factors but does not decompose them.Source: D Classical no-reset replay and… · p. 28, Table 20 · p. 28, Table 21 · p. 29

No-learner analysis

  • With fixed training-block scales, the no-reset process crossed in 45/45 pooled runs and 4,320/4,320 single-horizon streams on frozen residuals.The median pooled first crossing was step 19 and median peak log-wealth was 430. The passage describes nominal Ljung–Box and Kolmogorov–Smirnov results as diagnostics, because real-stream sequences are dependent.Source: D Classical no-reset replay and… · p. 28, Table 22 · p. 29
  • Under trailing scales, frozen residuals crossed in 39/45 pooled runs, with median peak log-wealth 59.1, and in 2,991/4,320 single-horizon streams.At horizon 1, trailing scaling reduced crossings from 45/45 under fixed scale; on weather, the raw pooled stream crossed 3/9 times versus 9/9 for the adapter. Across windows of 96 and 1,000 steps, pooled crossings were 36/45 and 40/45, while single-horizon crossings were 2,663/4,320 and 3,218/4,320.Source: D Classical no-reset replay and… · pp. 28–29, Table 23 · p. 30

Refiner analysis

  • The ORCA refiner’s no-reset process crossed in 45/45 pooled runs and 9/9 on every stream, despite preserving causal gains.Its median peak log-wealth was 397 versus 430 for the frozen stream, and the deployed gate fired a median 145 times per run versus 131. Under trailing scaling, it crossed in 37/45 runs and 1/9 on weather; thinned to one origin in 96, it crossed in 24/45.Source: D Classical no-reset replay and… · p. 29, Table 24 · p. 30

E Divergence counts by variant and protocol

Under immediate reveal, the full gate diverges frequently, while delayed reveal sharply reduces those counts. Results across variants show that safer intervention depends on the variant and setting; clean weather and sustained-contamination outcomes also differ.

E Divergence counts by variant and protocol

  • 60 immediate-reveal full-gate divergences fall to 2 under delayed reveal; the eight delayed-reveal Kalman divergences cluster in specified weather and ETTm1 settings.Seven occur on weather at seed 1 under the 5% spike injector; the eighth is Huber-only on ETTm1 under packet-freeze.Source: E Divergence counts by variant and… · p. 31
  • The capped policy has 1 immediate-reveal divergence and none in clean cells, while leaving the spike check live during recovery yields 47 divergences.Both capped variants have 2 delayed-reveal divergences, at the same two weather cells as the full gate.Source: E Divergence counts by variant and… · p. 31
  • Under immediate reveal, soft reset diverges in 60 of 450 runs, boost in 1, and override in 0; each has 2 delayed-reveal divergences.On clean weather under delayed reveal, the variants’ final MSE ranges differ substantially across backbones; on the other four clean streams, their excess MSE over the ungated filter is generally small.Source: E Divergence counts by variant and… · p. 31
  • Across 20 delayed-reveal weather seeds, soft reset exceeds the ungated filter in every seed on all three backbones, while override and capped policy beat it on Chronos-2 in every seed.The soft-reset versus full-gate comparison is significant after the stated Bonferroni factor only on TiRex.Source: E Divergence counts by variant and… · p. 31
  • Across the saved campaigns, 284 divergence records concentrate around the weather transient and a second traffic event; only 9 occur at the guard’s first check, step 200.The weather transient accounts for 181 records at step 18,536, 15 at 18,537, and 10 at 18,538.Source: E Divergence counts by variant and… · p. 31

F Full degradation tables

  • Under sustained contamination, every variant stays near zero in weather under both injectors, while immediate-reveal ETTh2 level-shift results indicate degradation across all 13 tabulated variants.For the fixed cutoff, the packet-freeze row ranges from +1.72 to +13.76.Source: F Full degradation tables · p. 32

G Adapter gain over the frozen backbone and its retention

The adapter can improve on the frozen backbone under immediate reveal, but its value depends on the stream and feedback protocol. Tested gates and fallback responses have mixed results: some reduce losses or divergences, while others degrade forecasts under contamination.

G Adapter gain over the frozen backbone and its retention

  • Under causal delayed reveal, the adapter is worse than the frozen backbone on every cell, while immediate reveal improves error on 12 of 15 cells but multiplies it 2.9 to 22 times on weather.The main campaign’s immediate weather errors remain above the frozen backbone’s; a separate Appendix H variant with a pinned forgetting factor and held noise scale reaches a gain of 0.47.Source: G Adapter gain over the frozen… · p. 33
  • With 5 percent spikes, the capped policy retains 0.96 of clean gain, versus 0.71 for the full gate and −0.35 for the ungated filter.Under packet-freeze contamination, several response variants lose 2.8 to 8 times the gain; under feedback-vector shift, every variant loses about three times the gain.Source: G Adapter gain over the frozen… · p. 33, Table 31 · p. 35
  • On clean streams, gated variants retain 0.90 to 1.06 of gain, while the fixed cutoff retains 0.75; retention is undefined when the ungated filter has no clean gain.Retention is measured relative to the ungated filter’s clean gain, so zero means no better than never adapting; weather and delayed cells are excluded.Source: G Adapter gain over the frozen… · pp. 32–33
  • The full gate’s divergences fall from 60 to 2, but delayed reveal still has eight Kalman divergences, seven in one weather spike-injection cell family.The remaining cases include two full-gate, one ungated-filter, three Huber-only, and two fixed-cutoff divergences.Source: Figure 7 · p. 34, Table 30 · p. 34
  • The first fallback mixture misses its clean-data rule and retains 0.46 of gain under spikes; the repaired version retains at least 0.90 on all 36 immediate-reveal runs.The repaired mixture keeps only 0.03 of median gain under spikes and loses 2.71 times the gain under level shift.Source: G Adapter gain over the frozen… · pp. 33–34

H Causal reveal, residual predictability, and the adapter’s forgetting

Causal reveal and composite coordinate feedback expose limits in the adapter’s forecasting value and stability. Forgetting-factor and noise-scale choices affect gains and divergences, while a fallback mixture limits some regime-shift losses.

H Causal reveal, residual predictability, and the adapter’s forgetting

  • Under composite coordinate reveal, the adapter without features is worse than the frozen backbone on 14 of 15 cells, with median gain −0.34; under delayed reveal it loses on all 15.Immediate reveal instead yields positive gain on 12 of 15 cells.Source: H Causal reveal, residual… · p. 35
  • The widened residual-feature variant diverges in 85 of 90 clean seeds, and its five surviving seeds finish 2.1 to 28.5 times the frozen error; wrapped in the fallback mixture, its median gain is −0.010.Its prespecified clean-gain rule passes in 0 of 15 cells.Source: H Causal reveal, residual… · pp. 35–36
  • The learned forgetting factor is 0.949 to 0.997 on non-weather streams and at its 0.90 floor on weather, corresponding to an effective memory of 20 to 330 origins.Learned observation noise shrinks to 0.07 to 0.59 of its starting value on several streams under immediate reveal, but grows 1.5 to 1.9 times on weather under composite coordinate reveal.Source: H Causal reveal, residual… · p. 36
  • Pinning the forgetting factor at 0.999 raises clean gain on all 12 non-weather cells and removes every divergence across 270 runs, but the prespecified rule still fails in 4 of 12 cells.The factor alone does not erase weather losses: under composite coordinate reveal, error remains 2.9 to 21 times frozen error.Source: H Causal reveal, residual… · pp. 36–37
  • Holding process noise alone changes non-weather gains by −0.06 to +0.11 and leaves all nine weather divergences; holding observation noise alone does not help on median.Holding both scales with the factor pinned yields a pooled gain of −0.13 and reaches within −0.05 of the frozen backbone on 4 of 12 non-weather cells.Source: H Causal reveal, residual… · pp. 36–37
  • On non-weather cells under a magnitude-1.0 regime shift, the adapter’s post-onset error is 0.16 to 0.73 of frozen error, but on weather it is 165 to 323 times higher under composite coordinate reveal.The fallback mixture caps weather’s ratio at 1.05 to 1.07 under composite coordinate reveal and 1.02 to 1.18 under delayed reveal.Source: H Causal reveal, residual… · p. 38

I ORCA diagnostic and simplified ports, not head-to-head method rankings

The ORCA diagnostic passes its prespecified delayed-reveal rule on the non-weather cells, but the campaign is scoped to particular streams, context, inputs, and implementation choices. The results bound the Kalman adapter’s loss rather than ranking published methods.

I ORCA diagnostic and simplified ports, not head-to-head method rankings

  • The five-refiner campaign uses cached forecasts, 96-step contexts, and complete horizon-length targets; refiners and Kalman variants differ in warm-up data and validation-label access.The refiners consume identical information under composite coordinate and delayed reveal, reproducing forecasts bit for bit on 225 of 225 runs.Source: I ORCA diagnostic and simplified… · pp. 38–39
  • The ORCA refiner’s median gain is nonnegative on all 12 non-weather cells under delayed reveal, with pooled gain 0.14 over 15 cells and no diverged seed.By the prespecified reading, this scopes the observed loss to the Kalman adapter family on these cells, horizon, context, reveals, and defaults.Source: I ORCA diagnostic and simplified… · p. 39, Table 44 · p. 42
  • Under delayed reveal, ELF, TAFAS, DSOF, and ridge reach nonnegative gain on 11, 8, 8, and 12 non-weather cells, respectively; the campaign does not rank published methods.The ridge refiner’s gains and the Kalman comparison do not isolate differences in estimator, features, and penalty.Source: I ORCA diagnostic and simplified… · pp. 39–40
  • The evaluation uses 96-step context while ORCA reports at 520, forecasts and adapts only one target channel, and gives refiners context plus cached forecasts unlike the Kalman adapter.A warm-up check leaves these input differences unseparated.Source: I ORCA diagnostic and simplified… · p. 40
  • Reported percentages use a different scale from ORCA’s, and released learning rates, penalties, and combiner rate were selected for that other scale.The campaign therefore does not explain the different reported profile for the ORCA residual refiner.Source: I ORCA diagnostic and simplified… · pp. 40–41

J No-parameter-drift simulation, per-run

In a fixed-design simulation with no parameter drift, the full adaptive pipeline fired in every run, while a fixed-construction null rarely fired. Matched controls implicate online hyperparameter learning in the repeated alarms.

J No-parameter-drift simulation, per-run

  • The simulation ran the full Kalman-filter, calibration, martingale, and response pipeline on a time-varying design with i.i.d. noise and no parameter drift.It used 60 independent runs of 20,000 steps, with a 500-step warm-up.Source: J No-parameter-drift simulation, per-run · p. 42
  • 60 of 60 runs fired at least once, averaging 37.9 fires per run, while the fixed-construction null fired in at most 1 of 60 runs.The first post-warm-up fire arrived 18–67 steps into the run.Source: J No-parameter-drift simulation, per-run · p. 42
  • The isolation campaign varied filter updates, hyperparameter learning, responses, stream permutation, calibration, resets, warm-up, and initial noise scale.It compared 22 variants at the same 60 seeds and 20,000 steps, except for the specified longer warm-ups.Source: J No-parameter-drift simulation, per-run · pp. 42–43
  • On the same permuted input, the fully learning variant fired in 60 of 60 runs, versus 1 of 60 when all three online hyperparameters were held fixed.The learning variant averaged 36.1 fires per run; the held variant averaged 0.02.Source: J No-parameter-drift simulation, per-run · pp. 42–43

K E-detector replay

Replaying alternative sequential detectors on saved clean streams did not remove alarms, while dependence-calibrated CUSUM behavior varied with calibration choices. The replays also show that guarantee conditions and detector scope matter.

K E-detector replay

  • On synthetic null checks, restarted alarm counts stayed within heuristic T/A references, but those references are not formal bounds on alarms after restarts.The Shiryaev–Roberts detector alarmed in 15 of 30 i.i.d.-score runs at A = 10^4.Source: K E-detector replay · p. 44
  • The replayed e-detectors inherit invalidity from the deployed p-value stream; the paper identifies exchangeability restoration or dependence-aware evidence as possible remedy directions.E-processes that test exchangeability directly are proposed as a way to measure whether a candidate remedy restores it.Source: K E-detector replay · p. 44
  • Short-reference block bootstrap and large-bandwidth Newey–West calibrations were anti-conservative on i.i.d. checks, whereas block length 1 and bandwidth 96 stayed within the stated tolerance.Bandwidth 96 also stayed at or below 0.01 on the tested AR(1) and 96-term moving-sum streams.Source: K E-detector replay · p. 44
  • On 180 saved streams, the i.i.d.-calibrated CUSUM alarmed in 180 of 180 runs at a median of 167 steps, while dependence calibration alarmed in 144 of 180 at 5,076 steps.The dependence-calibrated detector alarmed in 36 of 36 runs on electricity, ETTm1, traffic, and weather, but 0 of 36 on ETTh2.Source: K E-detector replay · pp. 44–45
  • WATCH and the plain conformal martingale crossed in all 180 saved shadow runs, and WATCH had often crossed before covariate reweighting began.WATCH crossed before adaptation in 141 runs; forced adaptation at step 250 still left 180 of 180 crossings.Source: K E-detector replay · p. 45

L Firing and response under a sustained regime change

Under sustained regime shifts, alarm behavior and protective responses varied: alarms did not reliably distinguish shifts from contamination, and some mitigations reduced responsiveness. A synthetic drift study likewise found slower, worse adaptation with the gate.

L Firing and response under a sustained regime change

  • Most deployed variants showed little onset-specific firing above background, and detector-stride-96 variants rarely fired within 96 steps of the change.Every-origin gates under delayed reveal fired within 96 steps in 0.18–0.20 of runs against backgrounds of 0.60–0.76; stride-96 median latencies ranged from 640 to 8,437 steps.Source: L Firing and response under a… · p. 46, Table 48 · p. 47
  • Spike checks that protect against isolated contamination can also suppress updates during persistent shifts, increasing post-onset error for several gated variants.Under delayed reveal, every gated variant had higher post-onset error than the ungated filter; the fixed cutoff reached 3.32 times its error at magnitude 3.0.Source: L Firing and response under a… · p. 46
  • No theoretical guarantee was established for sequential constructions on the shadow adapter’s saved scores, and the stride-96 constructions did not cross within 960 steps in any of 45 runs.The classical no-reset wealth process sometimes crossed soon after onset, but the clean-twin rates were similar over 960 steps.Source: L Firing and response under a… · pp. 46–47, Table 49 · p. 47
  • The adapter absorbed and overshot the shift, leaving at most one elevated score per 96-step interval, while frozen-backbone residuals remained elevated for much longer.At magnitude 1.0, frozen residuals stayed 14–17 times their pre-onset mean in every 96-step block to 2,000 steps; at magnitude 3.0, they were about 130 times higher.Source: L Firing and response under a… · p. 47
  • At magnitude 1.0, the monitor alarmed after onset in 43 of 45 runs under both reveal protocols, versus 8 of 45 clean twins, but the responses traded off level-shift and regime-shift error.Holding the adapter cost 1.14 and 1.10 times ungated error under level shift; handover to the frozen backbone reduced that cost to 0.667 and 0.704 but raised regime-shift cost to 6.66 and 1.99, respectively.Source: L Firing and response under a… · p. 48

M Synthetic drift latency

  • On the synthetic drift benchmark, the full gate was slower than the ungated filter on all three seeds and had roughly four times worse test-block MSE.The detector-stride-96 gate was slower still, with mean latency 162–198 steps versus 128–155 for the full gate.Source: M Synthetic drift latency · p. 48, Table 52 · p. 49
Loading 2610.07038v1…