Delayed Optimizer-State Transport Shapes Short-Horizon Training Decisions

Jinhui Guo

Source-linked AI summary

Delayed Optimizer-State Transport Shapes Short-Horizon Training Decisions

Jinhui Guo

arXiv:2608.24593v1Aug 25, 2026cs.LGphysics.comp-ph

TL;DR

The paper asks whether optimizer memory changes finite-horizon training decisions. It differentiates complete model–optimizer trajectories on specified short paths and finds that delayed transport affects schedule selection, response prediction, and shortlist quality.

Problem
Adaptive optimizers retain gradient history, but how this internal state changes finite training decisions has been unclear.
Method
The study differentiates specified eight-step AdamW paths through the complete model–optimizer state, selects exposure-matched schedules, and evaluates locked actions independently across Math–Code and Ising–CNN systems.
Results
Full transport improved held-out decisions, reordered schedules, and showed that preferred actions depend jointly on optimizer state and near-future minibatches; Adam moments formed a measurable transmission channel.
Takeaways & Limitations
On specified short paths, optimizer memory and near-future data order are actionable components of the training state, distinguishing local response prediction from finite-amplitude action selection.
Takeaways & Limitations
The analysis assumes a specified near-future minibatch sequence; unknown future drives would require averaging over possible paths, while sample reallocation could add gradient-covariance effects.

Abstract

from arXiv · show

Adaptive optimizers retain gradient history in moment variables, allowing a local change in loss weighting to alter later updates. We examine whether this delayed transport is large enough to change prospective short-horizon decisions. On committed future-minibatch sequences, we differentiate eight-step AdamW trajectories through the complete model--optimizer state and select exposure-matched Math--Code loss schedules before independent evaluation. Across 12 unused 0.3M Transformer histories, full transport lowers token-disjoint loss relative to an optimizer-aware immediate derivative in 10/12 histories (mean benefit $4.71\times10^{-4}$; exact one-sided sign test, $p=0.0193$). The two controllers act equally often but select different schedules in 60/96 windows. Crossed checkpoint--future-path tests attribute this reordering to the interaction between optimizer state and near-future data, while an independent Ising--CNN experiment shows that deleting moment-state transport destroys accurate response prediction. Full-transport scores also concentrate exact-rollout winners in larger candidate libraries, focusing finite-amplitude evaluation on a shortlist. On these committed short paths, optimizer memory and near-future data order are therefore actionable components of the training state, providing a mechanism-based criterion for when finite-horizon rather than one-step intervention is required.

I. INTRODUCTION

Adaptive-optimizer memory makes short-horizon intervention a dynamical problem: loss-weight perturbations propagate through augmented model–optimizer states before affecting later objectives. The paper formalizes this pathwise response and evaluates schedule selection on locked future paths.

Motivation: Optimizer moments retain gradient history, so similar immediate interventions can rank differently under objectives evaluated several steps later.This makes intervention timing a dynamical question rather than a purely local one.
Response decomposition: The response decomposes into intervention injection, finite-time model–optimizer transport, and observable-specific readout.The decomposition supports moment-tangent deletion, immediate-versus-full transport comparisons, and schedule ranking.
State representation: The augmented state includes model parameters and optimizer variables, allowing the optimizer update to be treated as locally Markovian.AdamW moments, optimizer-clock dependence, and prescribed minibatches are included in the update map.
Pathwise response: A perturbation at source step s is transported by the ordered Jacobian product and projected onto a later observable.The source-time dependence of the intervening updates makes future minibatch order relevant to terminal response.
Experimental premise: Exposure-matched eight-step Math–Code schedules are selected on committed future-minibatch paths and locked before independent readout.Both domains are evaluated at every step, so schedules differ in temporal loss-weight placement rather than sample allocation.
Decision framing: The study uses the same source–transport–readout framework to connect local perturbations with finite schedule decisions.Source-time derivatives are contracted with schedules for finite-action selection.

B. Optimizer memory as a transport channel

The study isolates Adam moment coordinates as a transport channel for perturbations. Its eight-step horizon substantially samples first-moment relaxation but only weakly samples second-moment relaxation.

Moment ablation: Removing Adam-moment perturbations while retaining the nominal trajectory isolates their contribution to the transport factor.The resulting memory-deleted tangent provides a coordinate-level diagnostic of moment-mediated transmission.
First-moment mechanism: A control pulse leaves an exponentially decaying record in Adam’s first moment.The retention factors β1 and β2 govern decay of the first and second gradient moments, respectively.
State representation: Including moments and the optimizer clock in the augmented state makes AdamW updates Markovian; removing moments leaves history-dependent parameter dynamics.This is the structural basis for treating optimizer memory as a response-transmission channel.
Configuration: The AdamW configurations use β1 = 0.9 and β2 = 0.999.These retention factors determine the corresponding discrete e-folding times.
Horizon sensitivity: 9.5 and 999.5 steps are the approximate discrete e-folding times for the first and second moments, respectively.An eight-step horizon therefore samples a substantial fraction of first-moment relaxation but only a small fraction of the second-moment timescale.

C. Finite-action scoring

Finite-action scoring converts source-time response derivatives into comparisons among exposure-matched temporal schedules. Full transport evaluates terminal effects across the remaining committed window, whereas the immediate comparator stops at the next state.

Schedule construction: Each candidate redistributes a fixed total domain weight across several source times within an H-step intervention.The schedule vector determines temporal placement of the loss-weight perturbation.
Exposure matching: The neutral action is u = 0, while nonneutral exposure-matched schedules differ only in temporal reweighting.Equal aggregate domain weight does not imply equal scores when source-time response varies.
Derivative definitions: Full source-time derivatives measure terminal response to a perturbation at one source time, with no summation over source times.The schedule-level sum appears only after contracting all source-time contributions with a complete schedule.
Controller comparison: Full transport carries a source perturbation through the remaining specified updates, whereas the immediate derivative reads it at the next state.Their comparison isolates the effect of the remaining training window on action selection.
Finite-action evaluation: Tangent scores guide finite-amplitude action selection on committed future minibatches after a clipping-branch compatibility screen.The primary seven-action experiment executes the compatible tangent-selected schedule directly.

III. EXPERIMENTAL DESIGN

The experiments test pathwise transport in a paired-domain Transformer and an independently structured Ising–CNN benchmark. Prospective action selection, independent readouts, and crossed checkpoint–future-path audits separate optimizer-state and future-drive effects.

Prospective protocol: The response construction is tested prospectively in Math–Code and Ising–CNN, with future minibatch paths fixed before scoring and actions locked before evaluation.This design compares transport-based decisions under controlled future paths.
Math–Code Transformer: The primary system is a byte-level causal Transformer trained on paired Math and Code minibatches with matched byte-token budgets.Training and validation files are disjoint, and the primary model has 296,504 parameters.
Control windows: H = 8 updates define each control window, with six zero-sum schedules of amplitude a = 0.02 competing against a neutral action.Validation positions are split into controller, audit, and test thirds to separate selection from terminal evaluation.
Confirmation: Confirmation uses 12 held-out 0.3M histories and compares full transport with neutral training, VGA, and an optimizer-aware immediate derivative.The immediate comparator includes the current AdamW-step mechanics but stops before later state propagation.
Independent benchmark: The Ising–CNN benchmark changes the architecture, data-generating process, and scientific readout while retaining the optimizer-state perturbation.It uses a 252,642-parameter GroupNorm CNN trained on two-dimensional Ising configurations.
Crossed audit: The Ising analyses include optimizer-memory ablation, prospective control against a locked Binder anchor, and crossed checkpoint–future-path auditing.Independent checkpoint and future-tape variation separates state, future-drive, and interaction contributions to schedule preference.

C. Prospective protocol and statistical inference

Prospective, independently seeded histories and locked future paths establish the decision-level test: delayed transport is evaluated before held-out readout, with history-level inference. Full transport improves outcomes, changes selected schedules, and shows strong dependence on checkpoint–future-path interaction, although confirmation differs by scale.

Prospective protocol and statistical inference: 12 held-out 0.3M Math–Code histories yielded a 4.708 × 10−4 mean token-disjoint benefit for full transport over the optimizer-aware immediate controller.The history-bootstrap 95% interval was [2.660, 6.545] × 10−4, with exact one-sided sign-test probability 0.0193.
Prospective protocol and statistical inference: 60 of 96 windows selected different temporal schedules despite both full and immediate controllers intervening in every window.Their outcome difference therefore reflects schedule selection rather than intervention frequency.
Prospective protocol and statistical inference: The 1M extension preserved one-window tangent accuracy but produced a positive mean and 9/12 positive histories in repeated action selection, below the prespecified 10/12 criterion.The authors treat decision-level confirmation as coming from the 0.3M cohort rather than the 1M extension.
Future-path conditioning: The crossed Ising design attributed 71.9% of centered schedule-score variance to state–tape–schedule interaction.The result indicates that the effect of a future-minibatch sequence depends strongly on its starting checkpoint.
Future-path conditioning: With committed future minibatches, full control improved 60/72 Ising cells and all six history means, whereas leave-one-tape-out state-only control had a −1.61 × 10−5 mean benefit.Averaging over other tapes suppresses the path-specific component needed to rank actions on the held-out tape.
Future-path conditioning: The tested control state therefore comprises the augmented optimizer state and specified near-future drive, not the checkpoint alone.The paper uses this boundary to distinguish path-conditioned control from state-only action prediction.

C. Optimizer-moment contributions to the response

Full-state transport identifies Adam moments as a measurable, path-specific response channel and improves finite-action screening across the tested Transformer and Ising–CNN settings.

Transformer response reconstruction: NRMSE/cosine at 0.3M are 0.0239/0.9997 for m and 0.0490/0.9988 for e, confirming accurate full-state response reconstruction.At 1M, the corresponding pairs are 0.0138/0.9999 and 0.0304/0.9995.
Optimizer-coordinate contributions: The full tangent outperforms memory deletion in all 62 eligible momentum and AdamW-family cells.Under SGD, the common nominal pulse produces a larger finite response and inaccurate local reconstruction.
Path specificity: Individual-path transport remains accurate despite weak ensemble-mean transfer, with cosine similarity 0.083 for the averaged e profile and NRMSE = 0.0304 on individual 1M e paths.This contrast indicates that the response depends on the realized path rather than a scale-independent averaged kernel.
Cross-system validation: Moment-state deletion raises Ising–CNN reconstruction NRMSE from 0.002115 to 0.6923 for m and from 0.001580 to 0.6871 for e.The degradation supports moment-coordinate transmission under a different architecture and physically anchored readout.
Finite-action screening: Winner recall for full, memory-deleted, immediate, and random ranking is 0.889, 0.796, 0.370, and 0.296 on Ising, versus 1.000, 0.907, 0.407, and 0.155 on Math–Code.At equal exact-rollout budget, full transport most often places exhaustive finite-amplitude winners near the top.
Scope: The evidence is strongest for eight-step control with Adam-like optimizers and the 0.3M Transformer, while future-path marginalization and longer horizons remain direct tests.At 1M, the mean decision effect remains positive but accompanies accumulated finite-amplitude and closed-loop deviations.

VI. CONCLUSION

The paper asks how optimizer memory changes finite training decisions, differentiates locked eight-step AdamW paths through the complete training state, and finds improved prospective decisions with path-dependent schedule selection.

Adaptive optimizer state changes finite training decisions, but the role of retained gradient history had remained unclear.
The study differentiates specified eight-step AdamW paths through the complete model–optimizer state, selects exposure-matched schedules, and locks actions before independent evaluation.Math–Code provides the primary test, with Ising–CNN testing the mechanism under a different architecture, data-generating system, and anchored readout.
Full transport improves held-out decisions over an optimizer-aware immediate derivative and reorders schedules despite matched intervention frequency.Crossed checkpoint–future-path experiments link preferred actions to the current optimizer state and near-future minibatches.
Full-state derivatives serve as mechanistic probes and screening scores for finite interventions along committed paths.The conclusion identifies short-horizon loss-weight control as a path-conditioned problem on augmented training state.

A.1. AdamW tangent equations

The AdamW tangent propagates perturbations through clipped gradients, exponential moments, bias correction, and parameters, while coordinate deletion isolates moment-state transmission. Exposure-matched schedules and locked readouts provide the experimental setting for comparing these transport-aware decisions.

State and update equations: The AdamW state includes parameters, clipped-gradient moments, bias-corrected moments, weight decay, and denominator stabilization.The moments track the clipped gradient and its elementwise square; η_t, λ, and ϵ govern the parameter update.
State and update equations: The tangent equations propagate loss-weight perturbations through gradient, moment, and parameter updates on a fixed differentiable clipping branch.The paired-domain control enters through the loss-gradient contrast, while the clipping differential and Hessian–vector products are evaluated matrix-free.
Coordinate-resolved transport: Coordinate-resolved transport attributes the next parameter tangent mainly to parameters and first moments, with mean contributions of 0.7930, 0.2089, and −0.00187 at 0.3M.The corresponding 1M contributions are 0.7936, 0.2068, and −0.00037, and their near-unit sums provide an internal closure check.
Coordinate-resolved transport: Memory deletion removes propagated moment tangents while preserving the nominal trajectory, separating moment-coordinate transmission from the immediate one-step comparator.The immediate comparator includes the complete one-step tangent but stops before later state propagation.
Math–Code setup: The experiment uses exposure-matched zero-sum eight-step Math–Code schedules whose temporal order changes without changing aggregate domain weighting.Controller, audit, and token-disjoint test partitions separate action selection from post-lock evaluation.
Scale and duration: At 1M, the mean 64-step benefit is 2.329 × 10−4 with 9/12 positive histories, while independent-test benefit approaches zero after 224 controlled steps.The study therefore distinguishes accurate one-window response from less uniform repeated closed-loop utility.

B.2. Ising–CNN system and physical anchor

The Ising–CNN system uses domain-separated temperature anchors and a differentiable neural crossover as its controller readout. A Binder-cumulant crossing supplies the physical reference used to calibrate the neural anchor.

Ising data domains: The Ising domains separate near-transition and far-from-transition temperature anchors while keeping supervised class labels distinct from controlled data domains.Domain A contains 2.10, 2.18, 2.36, and 2.44; domain B contains 1.50, 1.80, 2.80, and 3.20.
Neural crossover readout: The neural controller readout is the temperature b_TN where a fitted linear classifier preference changes sign.The fit uses ordered- and disordered-class logits at temperatures 2.18, 2.24, 2.30, and 2.36.
Physical anchor: The Binder construction estimates an extrapolated crossing temperature T_B from ratio-two crossings across multiple system sizes and correction exponents.The Monte Carlo bank uses eight chains, 256 configurations per temperature, eight temperatures, and sizes L ∈ {8, 12, 16, 24, 32, 48}.
Physical anchor: The calibration objective penalizes neural displacement from the Binder anchor, variation across neural chains, and poor conditioning of the fitted crossing.Its fit-quality term has relative coefficient 0.1, with 10−12 used for stabilization.

B.3. Crossed future-tape and specified-path designs

The designs test whether schedule decisions depend jointly on the current optimizer state and committed near-future minibatches. Crossed audits decompose these influences, while specified-path evaluation locks predictions and actions before independent testing.

Crossed future-tape design: The crossed audit uses four histories, three checkpoints, eight future tapes, and 96 state–tape cells at amplitude a = 0.002.Each future tape is a pre-generated sequence of eight paired near- and far-domain minibatches.
Crossed future-tape design: The crossed contrast decomposes schedule scores into state, tape, and state–tape–schedule interaction components.Each component is broadcast across the full array before squared sums produce the reported percentages.
Crossed future-tape design: Leave-one-tape-out comparisons include neutral, memory-deleted, immediate, random matched-weight, and globally fixed schedule controls.Actions are formed from the other seven tapes and evaluated on the held-out tape.
Specified-path design: The specified-path design conditions each decision on its committed eight-minibatch tape, yielding 72 cells across six histories, three checkpoints, and four tapes per checkpoint.Predictions, actions, branch records, checkpoints, and code are locked before generating an independent test chain.
Response estimands: Finite responses compare perturbed and nominal trajectories sharing identical minibatches, and central secants approach source-time derivatives as amplitude decreases.A transported tangent can remain nonzero even when the chosen observable’s measured response decays.
Response estimands: Response quality is measured by NRMSE and cosine alignment, while action agreement and exhaustive-winner recall assess candidate ordering.These measures distinguish response reconstruction from finite action selection.
Statistical methodology: History-level decision effects use independent histories as inferential units, with bootstrap intervals and exact sign tests applied to history effects.Within-chain and contiguous-token dependence is retained through block resampling.
Statistical methodology: Winner recall pools 54 Ising and 108 Math–Code settings but is reported as an empirical calibration diagnostic because settings share paths and histories.The passage explicitly states that these settings are not exchangeable calibration units.

SUPP. D. FINITE ACTIONS, BRANCH COMPATIBILITY, AND EXACT RERANKING

Finite-amplitude schedules are ranked by transported tangent scores, screened for clipping-branch compatibility, and exactly reranked when candidate scores are too close. An adaptive shortlist corrects large-library misses while limiting exact rollouts.

Finite-action ranking: The state-error bound isolates intervention size, accumulated candidate–neutral separation, and subsequent Jacobian amplification as sources of finite-amplitude error.This dependence motivates small amplitudes and exact reranking of close candidates.
Branch compatibility: A homotopy from neutral to each candidate screens for changes in the sampled gradient-clipping branch and requires distance from the unit clipping threshold.Response reconstruction is evaluated within a fixed smooth branch after this screen.
Finite-action ranking: Tangent scores rank finite-amplitude candidates, while candidate-dependent remainders determine when the tangent ordering is reliable.If the score gap is at most 2ϵ, finite-amplitude comparison is required.
Exact reranking: In development libraries, q = 8 gives exact-action agreement of 15/18, 16/18, and 13/18, while q = 16 gives 17/18, 17/18, and 18/18.Misses occur when the exhaustive winner falls outside the retained shortlist.
Shortlist calibration: The shortlist retains candidates within an empirical tolerance based on candidate count and perturbation amplitude, then exactly evaluates the retained actions with neutral.The rule is an empirical calibration rather than a direct split-conformal guarantee.
Shortlist calibration: Adaptive calibration reaches 18/18 agreement in each fresh library, correcting the two large-library failures that fixed q = 16 leaves unresolved.The adaptive shortlists expand to 43 and 44 candidates on the affected K = 128 cases.

SUPP. E. ADDITIONAL EVIDENCE FOR RESPONSE, CONTROL, AND RANKING

Across specified short paths, full model–optimizer transport accurately predicts delayed responses, improves locked schedule decisions, and materially reorders action rankings relative to an optimizer-aware immediate derivative.

Response accuracy: NRMSE values of 0.0239–0.0490 at 0.3M and 0.0138–0.0304 at 1M accompany cosine similarities of 0.9988–0.9999 for full-state tangent responses.These results hold for both antisymmetric and symmetric readouts on specified eight-step paths.
Response accuracy: Deleting moment-coordinate tangents preserves the first lag but removes most later response transmission in the Ising–CNN benchmark.Moment deletion NRMSE rises to 0.6923 for both readouts at later lags.
Prospective control: Full exceeds VGA in all 12 histories by 7.7192 × 10^-4 on average, while full-minus-neutral improvement averages 7.8052 × 10^-4 across 11/12 positive histories.Both prespecified full–immediate and full–neutral criteria pass.
Ranking: The full and immediate controllers choose different schedules in 60/96 states despite acting in every window, reflecting reordered temporal schedules under matched intervention frequency.At common initial states, score changes already occur before closed-loop trajectories diverge.
Ranking: 71.9% of centered schedule-score variation is state–tape–schedule interaction, and full actions beat immediate actions in 57/61 informative realized-tape cells.Averaging over other tapes can cancel the preference relevant to the realized path.
Future-path confirmation: With fixed future paths, full actions improve 60/72 cells and all six history means, beating immediate actions in 30/41 informative cells.Validation-to-test contrast calibration is imperfect, but selected actions retain positive test utility.

E.4. Equal-budget winner recall, transfer, and cost

Transport-guided ranking recovers exhaustive winners in larger candidate libraries while reducing exact rollouts, but calibration and empirical scope impose clear boundaries on recall and cost claims.

Winner recall: At q = 8, full ranking recalls 48/54 Ising and 108/108 Math–Code exhaustive winners, outperforming memory-deleted, immediate, and random rankings.This ordering holds for every tested q ∈ {1, 2, 4, 8, 16}.
Transfer: Fixed q = 16 recovers all 108 fresh Math–Code exhaustive winners, while the adaptive rule recovers 100/108 across three matched libraries.Adaptive recall is 32/36 in sphere A and 34/36 in each remaining library.
Cost: Math–Code adaptive screening evaluates six paths per bundle after calibration, compared with 327 candidate paths in the largest-library bundle.The saved exact paths eventually exceed the one-time calibration count when library structures are reused.
Cost: The adjoint takes 0.80 times the immediate controller’s total time, while forward full takes 1.33 times that total and the adjoint reaches 2.72 times its peak RSS.The complete per-window prospective comparison is roughly 10.7 times ordinary training and readout.
Comparator boundaries: The early full–VGA difference changes sign across histories, although full transport improves over neutral on average.VGA combines validation-gradient alignment with current AdamW geometry, whereas full transport also includes delayed propagation.
Scope and limitations: Longer 1M closed-loop runs show lower history consistency because repeated decisions change later Jacobians and nonlinear remainders accumulate.Accurate individual eight-step derivatives therefore do not guarantee consistent long-suffix control.
Scope and limitations: Leave-one-history-out recall is 44/54 for Ising and 97/108 for Math–Code, and the shortlist threshold is treated as an empirical risk budget without finite-sample coverage.Within-history dependence affects calibration, especially when empirical quantile indices approach the sample maximum.