Beyond the Clock: Measuring the Value of Adaptive Revision

Ayushi Chadha

Source-linked AI summary

Beyond the Clock: Measuring the Value of Adaptive Revision

Ayushi Chadha

arXiv:2609.00874v1Sep 1, 2026cs.AI

TL;DR

The paper asks whether state-conditioned supervisory revision adds value beyond fixed timing in hierarchical latent reasoning. It evaluates matched adaptive controllers, frozen timing interventions, exhaustive schedules, and counterfactual diagnostics. Across seeds and analyses, state dependence and behavioral diversity did not produce reproducible outcome gains beyond strong fixed policies, while position dominated some predictive evidence.

Problem
The paper examines whether higher-level controllers can assign limited revision decisions better than fixed timing, and how to distinguish state dependence from useful adaptive value.
Method
The study combines three-seed matched-budget adaptive audits, frozen-checkpoint timing interventions, exhaustive K = 2 schedule enumeration, and counterfactual PERSIST/REPLAN diagnostics.
Results
None of the three learned adaptive policies outperformed the best forced timing policy; pass position explained 79.464% of critic prediction variance, while pass-only prediction reached pooled AUROC 0.966.
Takeaways & Limitations
Adaptive control should be evaluated separately for state dependence, behavioral change, and outcome value beyond a strong non-adaptive policy.
Takeaways & Limitations
The exhaustive sweep, oracle analysis, and counterfactual diagnostic are seed-0 analyses in a mechanism-oriented ConceptARC-mini setting, leaving broader tasks, architectures, and compute regimes untested.

Abstract

from arXiv · show

As agentic systems become compound systems, increasingly important decisions move above task execution itself: when should a higher-level controller preserve the strategy guiding another process, and when should it revise it? We study this meta-level control problem in a hierarchical latent reasoner whose manager can retain or replace a commitment governing lower-level computation. Across three precommitted training seeds, learned revision timing produces qualitatively different policies, ranging from an almost deterministic early clock to substantially more state conditioned schedule distributions, yet none outperforms the best forced timing policy evaluated on the same frozen checkpoint. This separates state dependence from decision value: a controller can vary its actions with internal state without turning that variation into a reproducible task-performance benefit. A deeper intervention study on the original checkpoint shows that timing itself is consequential and order-sensitive, while exhaustive enumeration reveals that a strong fixed schedule captures most of the measurable value available from timing at this decision budget. Counterfactual PERSIST/REPLAN diagnostics further show why score-level evidence can be misleading when predictability is dominated by decision position rather than within-position discrimination. Together, these results argue that learned meta-level control should be evaluated along three separate axes: whether its score depends on state, whether that dependence changes realized behavior, and whether those changes capture outcome value beyond a strong non-adaptive policy.

1 Introduction

The paper isolates when a higher-level controller should preserve or revise a latent commitment during lower-level computation. Across precommitted seeds, learned revision timing varied substantially but did not deliver reproducible value beyond strong fixed timing.

A manager decides whether to retain or replace a directional latent commitment that guides a faster worker across refinement passes.
Across three precommitted seeds, adaptive policies ranged from an early clock to diverse schedules, yet none outperformed the best forced timing policy on its frozen checkpoint.
The central distinction is that state dependence, changed behavior, and improved outcomes are separate properties of supervisory control.
The study combines replication, frozen interventions, exhaustive timing enumeration, and counterfactual branching as a diagnostic protocol for meta-level control.

2 Supervisory Replanning in a Hierarchical Reasoner

The testbed uses a two-level latent reasoner in which a persistent manager-generated commitment guides worker refinement. Causal timing rules ensure that revisions affect only subsequent worker passes, while the adaptive trigger uses detached state features to decide whether to replace the commitment.

A slow manager proposes a persistent directional commitment, while a fast worker performs task-conditioned refinement that consumes it.
A commitment emitted after pass s remains active across later worker passes until the next intervention, with dwell defined by the number of consuming passes.
With M = 8 passes, a revision decided after pass m is first consumed on pass m + 1; optional revisions are eligible only after passes 2–7.
For K = 2, fixed schedules are [1, k] for k ∈ {2, 3, 4, 5, 6, 7}, making emission order and persistence directly comparable.
The adaptive manager replaces a fixed pass-based clock with a detached six-feature state summary and a Bernoulli retain/replace policy.
Training reconstructs a local value-preserving Jacobian at the first consuming pass, providing local credit without unrestricted backpropagation through the full trajectory.

3 Experimental Protocol and Controlled Evaluation

The evaluation separates reasoner quality, intervention budget, and timing using matched adaptive and fixed controls. It calibrates intervention counts without task performance, then tests timing on frozen checkpoints through controlled interventions and exhaustive schedule comparisons.

The protocol first establishes fixed-clock controls, then trains a matched-budget adaptive controller and intervenes directly on timing after freezing parameters.
Primary evidence consists of a 400-step three-seed adaptive audit, frozen-checkpoint schedule intervention, and preregistered exhaustive K = 2 sweep.
P = 6 exceeded P = 4 on all three seeds, with a mean held-out token-accuracy difference of 0.00248 despite modest overall effects.
Threshold calibration used intervention counts only, yielding K̄_cal = 2.000000 and K̄_final = 2.000814 on the ordered 3,686-episode final split.
The strongest timing control freezes the adaptive step-400 checkpoint and changes only the position of its second commitment.
Because M = 8 and K = 2, all six legal schedules can be evaluated on the same frozen checkpoint and ordered episodes.

4 Learned Replanning Is State-Dependent but Not Reliably Better

Across three seeds, learned revision timing produced qualitatively different policies but never beat the best forced schedule on the same frozen checkpoint. Frozen interventions and exhaustive timing sweeps show that timing—and its order—matters, while the learned policies fail to capture that value reliably.

Policy diversity: Seed 0 was nearly deterministic, while seeds 1 and 2 produced substantially different schedule distributions and score structures.Seed 0 used [1, 2] in 99.19% of episodes with 0.0776-bit entropy; seed 1 had 2.8598 bits of entropy, and seed 2 mostly used [1].
Cross-seed comparison: Adaptive timing lost to the best tested forced schedule on every frozen checkpoint, by 2.0496, 0.3386, and 0.0568 percentage points for seeds 0–2.Mean adaptive accuracy was 47.6037%, versus 48.4187% for the best within-checkpoint forced schedule; the mean is descriptive across three seeds.
Frozen intervention: Forced later interventions on the same seed-0 weights raised accuracy from 0.467002 with [1, 2] to 0.487511 with [1, 4] and 0.486622 with [1, 6].These changes used the same K = 2 budget and required no retraining, isolating the controller’s timing choice.
Timing landscape: The exhaustive [1, k] sweep peaked at [1, 5] with 0.521206 micro accuracy, indicating a narrow, non-monotonic timing landscape.The learned policy concentrated on an early region identified as poor by the controlled endpoint measurement.
Order effects: Reversed schedules differed despite matching dwell lengths: longer-first minus shorter-first gains were 0.00924, 0.00405, and 0.00097, with all 95% intervals excluding zero.Thus, where a duration occurs in the reasoning trajectory contributes to its value, not only the duration itself.
Interpretation: The evidence separates state dependence, behavioral variation, and outcome utility: schedule diversity alone did not yield reproducible performance gains beyond a strong fixed policy.The cross-seed result remains negative even though learned policy forms range from clock-like to highly diverse.

5 Auditing the Supervisory Signal

The audit separates score variation from useful within-position discrimination and endpoint timing. Although aggregate diagnostics show strong predictability, most of that signal is explained by decision position, while local revision value does not identify the globally best timing policy.

Score variation versus realized policy: 16.724% within-position score variance appears at an early checkpoint, but the calibrated step-400 policy executes [1, 2] almost universally.The early score remains dispersed enough to produce up to 21 schedules and 2.344 bits of schedule entropy at intermediate thresholds.
Matched counterfactual branching: 90.365% of counterfactual branches favor PERSIST, while a tiny critic still achieves held-out R2 = 0.406 and Spearman ρ = 0.729.These aggregate metrics suggest a strong state signal, but they do not establish useful within-position discrimination.
Conditioning on position: 79.464% of critic prediction variance is explained by pass position, and the residual target has held-out R2 = −0.007.A pass-only predictor already achieves reconstructed held-out R2 = 0.292, showing that pooled predictability is largely temporal.
Conditioning on position: A pass-only predictor reaches pooled AUROC 0.966 because all 12 REPLAN-favoring states occur at pass 2.Adding residual state information improves rare-state balanced accuracy, but the evidence is too sparse and position-concentrated to justify a new end-to-end controller.
Credit horizon versus endpoint value: The local diagnostic favors early replanning, whereas endpoint interventions show that later fixed timing is substantially better on the same checkpoint.The paper therefore rejects a pointwise claim that every locally favorable revision is globally harmful, retaining the narrower conclusion that local predictive signals need not yield correct endpoint timing.
Credit horizon versus endpoint value: One-consuming-pass credit reaches the trigger, but that local horizon need not value intervention on the horizon that ultimately matters.This bounded credit path was used as a stop rule rather than sufficient evidence for another adaptive training run.

6 How Much Is Adaptation Worth?

The paper asks how much episode-specific timing value remains after selecting a strong fixed schedule. Exhaustive evaluation finds that the best fixed clock captures most measurable timing value, with residual oracle gains concentrated in a small minority of episodes.

Motivation: The design question shifts from learning timing independently on every episode to choosing a strong global timing prior before pursuing residual adaptive value.This separates the possibility of learning the wrong timing policy from the possibility that little adaptive value remains.
Random, fixed, and oracle timing: The best fixed schedule is [1, 5] with Afixed = 0.521206, while an unreachable held-out-label oracle reaches Aoracle = 0.522557.The oracle selects the best schedule separately for each episode after evaluating all six counterfactual trajectories.
Residual opportunity: Only 281 of 3,686 episodes obtain any additional correct tokens over fixed [1, 5], and eight episodes contribute half of total micro headroom.Ties are pervasive: 88.2% of episodes share their maximum across at least two schedules, and 53.7% tie across all six.
Adaptation prerequisites: Adaptive control requires both heterogeneous decision value and substantial heterogeneity remaining after a strong non-adaptive baseline.Frozen interventions satisfy the first condition, but the narrow oracle margin and limited residual-state evidence weaken the second.
Decision-value decomposition: About 71% of the improvement from uniform-random timing to the held-out-label oracle is captured by the best fixed clock.After selecting a good clock, at most about 29% of the already-small timing interval remains for perfect episode-specific selection.

7 Discussion: From Internal Revision to Meta-Level Control

The paper frames internal revision as a controlled testbed for a broader meta-level control problem in compound AI systems. It argues that evaluation must distinguish state-dependent scores, changed decisions, and outcome value beyond strong non-adaptive policies.

Scope and transfer: The internal manager testbed is not an autonomous agent rewriting another agent or harness; its transferable object is the evaluation problem.The broader systems named in the paper also observe evidence, assign credit to revisions, and turn scores into actions.
Evaluation axes: Adaptive meta-level control should be evaluated separately for state-dependent scores, changed decisions, and added outcome value.The paper warns that score dependence or policy diversity alone should not be treated as progress.
Evaluation discipline: Frozen interventions, counterfactual branching, and strong non-adaptive baselines provide this evaluation discipline even when exhaustive enumeration is impossible.The lesson is methodological rather than a recommendation to use fixed clocks universally.

8 Limitations

The study’s broadest analyses are confined to seed-0 experiments in a compact ConceptARC-mini testbed. Several results therefore establish timing and order effects within a restricted schedule space rather than general laws across tasks, architectures, or compute regimes.

Scope: The adaptive audit covers three precommitted seeds, but exhaustive sweep, oracle, and counterfactual analyses are seed-0-only.Broader tasks, architectures, and compute regimes remain untested.
Experimental boundaries: ConceptARC-mini is a mechanism-oriented setting, and the K = 2 sweep covers six schedules on one anti-diagonal of dwell pairs.The sweep establishes order sensitivity, not independent dwell effects.
Interpretation boundaries: The held-out-label oracle is an unattainable upper bound, while micro and episode-averaged accuracy are reported as separate metrics.Residual trigger variation is not interpreted causally.

Responsible-Use Statement

The paper recommends evaluating learned supervisory control with matched-compute baselines and interventions that test realized decisions, while limiting broader conclusions to the studied evidence. Its analyses distinguish headline three-seed audits from seed-0 mechanistic studies and contextual controls.

Responsible-Use Statement: Matched-compute baselines and frozen interventions or counterfactual rollouts are recommended to test whether supervisory signals change downstream outcomes.Scores or diverse actions can appear adaptive without improving outcomes.
Evidence Scope: The headline evidence comes from three precommitted 400-step adaptive audits with within-checkpoint forced-schedule comparisons.The exhaustive sweep, oracle analysis, and counterfactual diagnostic are deeper mechanistic analyses of seed 0.
Schedule Notation: Schedule notation treats an emission after pass m as changing the commitment first consumed on pass m + 1.For M = 8, [1, k] contains the mandatory initial emission plus one replacement, with the two dwells summing to seven worker passes.
Calibration: Threshold calibration optimizes only mean intervention count on a calibration split before one frozen evaluation on a disjoint final split.Task labels and final accuracy do not select the operating point.
Bootstrap: Schedule and oracle comparisons use 10,000 paired episode-bootstrap resamples preserving identical episodes across clocks.Main-text confidence intervals are percentile intervals.

B Supplementary Quantitative Results

The supplementary quantitative results organize the evidence into three-seed adaptive-versus-forced controls, headline metric checks, and preregistered reversed-order comparisons. Together, these tables separate matched-control performance from order-sensitive timing effects.

Three-seed adaptive audit: Table 2 reports the three-seed adaptive audit alongside frozen-checkpoint forced controls.The comparison tests adaptive timing against forced schedules while retaining the matched audit structure.
Headline quantitative checks: Table 3 presents headline quantitative checks without mixing metrics across aggregation families.This preserves distinctions among the paper’s separately labeled quantitative measures.
Reversed-order comparisons: Table 4 reports preregistered reversed-order comparisons as longer-first minus shorter-first micro token accuracy.The corresponding schedule pairs test whether exchanging dwell lengths changes performance.

C Reproducibility Notes

The reproducibility notes specify the primary model configuration, optimization settings, adaptive seeds, and frozen-checkpoint integrity checks. They document both computational choices and safeguards used during schedule-control evaluation.

Model and objective: The primary model uses 32-dimensional hidden and goal states, two attention heads, one layer at each hierarchy level, and one cycle per level.The objective combines Stablemax token, halt/continue, and directional losses with λF = 0.05 and intervention weight η = 0.
Optimization and training: Training uses AdamATan2, batch size 4, learning rate 10^-4, puzzle-embedding learning rate 10^-2, weight decay 0.1, no warmup, and 400 optimizer steps.Execution is on CPU, and adaptive replication uses seeds 0–2.
Evaluation controls: Each adaptive step-400 checkpoint is frozen before schedule-control evaluation, with runtime fixed-compute completion asserted and terminal goal emissions suppressed.Counterfactual features and targets were detached during evaluation.
Checkpoint provenance: The three adaptive checkpoints are identified by recorded SHA-256 digests for seeds 0–2.These digests support exact checkpoint provenance across the replicated audits.