PaperlayerView exact revision on arXiv

Source-linked AI summary

Target-Independent Micro-Interventions for Predicting Training Response Across Language-Model Families

Zhongxuan Liu, Sicheng Zhou, Hongzhi Wang

arXiv:2609.08618v1cs.LG

TL;DR

Current benchmark scores do not determine how a checkpoint will respond to another training episode. The paper measures target-independent pulse responses as an L-State and uses direct and operator readouts to predict held-out-family responses. Both readouts improve over capability alone in development and sealed tests, with different readouts performing best across endpoints.

  • Problem

    Current capability scores describe present behavior but leave a checkpoint’s response to another training episode unresolved.

    Source: §1 · p. 1

  • Method

    Four standardized target-independent training episodes measure local capability changes, which are combined with current capabilities as an L-State for direct or operator readouts.

    Source: §1 · p. 1

  • Results

    Both pulse readouts reduce development MSE by 39.4% versus capability; on GLM, operator MSE reduction is 78.3%, while the Granite selector reaches RMSE 0.554 versus 1.172 for capability.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Target-independent pulse responses expose transferable training-response information beyond current capability, with direct and structured readouts covering complementary transfer regimes.

    Source: §6 · p. 10

    The transfer chain assumes a named pulse gauge, smooth local dynamics, spanning pulse excitation, positive source coverage, and bounded measurement and misspecification errors.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Benchmark scores describe what a checkpoint can do now, but they do not determine how it will respond to the next training episode. We measure this missing state by branching four short, standardized, target-independent micro-interventions from the same checkpoint and recording their effects in a common capability space. Together with current capability, these responses form L-State; its pulse block supports a flexible direct readout and a structure-preserving operator readout. Under smooth local dynamics, the operator construction admits an end-to-end cross-family bound with explicit source- and target-family coordinate heterogeneity. In three-family leave-one-family-out development, both pulse readouts reduce source-standardized MSE by 39.4% relative to capability alone, while separating the best response and direction estimates. On sealed GLM-4-9B, the direct and operator readouts reduce MSE by 71.8% and 78.3%, respectively, and the operator readout raises sign balanced accuracy from 0.366 to 0.754. On sealed Granite-3.1-8B, the direct readout reaches RMSE 0.544 and a development-fitted action-wise selector reaches 0.554, compared with 1.172 for capability alone. A five-family audit finds that the operator coordinate varies by action and family, and that modeling these deviations improves retrospective held-trajectory prediction. Target-independent interventions therefore expose training-response information that current capability misses, with direct and structured readouts covering complementary transfer regimes.

1 INTRODUCTION

Current capability scores leave a checkpoint’s response to future training unresolved; standardized target-independent interventions measure that response as an L-State. Pulse readouts improve held-out-family prediction, while theory and a five-family audit examine transfer structure and coordinate variation.

1 INTRODUCTION

  • Four standardized, target-independent training episodes measure local capability changes; concatenated with five current capabilities, their responses form a 25-dimensional L-State.Its 20-dimensional pulse block supports direct and operator-factorized prediction in a shared evaluation space.Source: §1 · p. 1, Figure 1 · p. 2
  • In leave-one-family-out evaluation, target-action responses remain sealed until predictions are frozen, testing transfer of response state across model families.The readouts use source-family target-action responses and may measure the held-out family’s current capabilities and standard-pulse responses.Source: §1 · p. 2
  • The direct readout learns a flexible map from pulses, while the operator factors local response geometry from a source-estimated action coordinate.Conditional theory supports the operator and gives guarantees with explicit source- and target-family coordinate-bias terms.Source: §1 · p. 2
  • Both pulse readouts reduce development source-standardized MSE by 39.4% versus capability alone; on GLM, direct and operator MSE reductions reach 71.8% and 78.3%.The operator has the highest sign balanced accuracy in development and raises it by 38.8 points over capability on GLM; a development-fitted selector reaches RMSE 0.554 on Granite versus 1.172 for capability.Source: §1 · p. 2
  • On Granite, the direct readout reaches RMSE 0.544, while the full L-State readout reduces action regret by 25.5%.The operator’s science prediction shows a reversed sign across 27 states and four additional realizations, with a shrunken and unstable source action coordinate.Source: §1 · p. 2
  • The five-family audit quantifies family and family–action deviations in pooled action coordinates and links this heterogeneity to readout selection.The paper’s evidence program also includes conditional theory, two sealed transfer tests, and an audit of coordinate deviations.Source: §1 · p. 2, Figure 2 · p. 3

2 RELATED WORK

Prior work predicts adaptation from probes, training traces, scaling laws, representations, or training-data effects. This paper instead measures target-independent checkpoint responses and evaluates signed candidate-action predictions across held-out model families.

2 RELATED WORK

  • Prior adaptation methods characterize current functional behavior, whereas L-STATE uses controlled training interventions to predict candidate-action response vectors.Related approaches include diagnostic probes, intermediate-checkpoint analyses, scaling laws, transferability scores, compact model representations, and training-free fingerprints.Source: §2 · p. 3
  • Datamodels, influence methods, and task vectors study effects of training data or completed updates; L-STATE identifies the checkpoint through target-independent responses in evaluation space.The paper asks whether pulse responses add usable information beyond current capability.Source: §2 · p. 3
  • The evaluation uses model-family identity as the outer split and selects on source families, then chronologically seals held-out target-action labels until predictions and choices are frozen.The intervention analogy draws on active system identification, while the local response model states its training-domain assumptions explicitly.Source: §2 · pp. 3–4

3 L-STATE AND ITS PULSE READOUTS

Current capability can leave checkpoints with different training responses indistinguishable, so the paper measures response with controlled pulses and uses their geometry to predict unseen-family behavior. It develops a bounded operator-transfer route while accounting for family and family–action coordinate heterogeneity.

3.1 WHY STATIC CAPABILITY IS INSUFFICIENT

  • Equal capability can conceal different training responses; controlled interventions make this within-level-set variation observable.An exact capability-only response map exists only when responses are constant on every capability level set.Source: §3.1 · p. 4

3.2 LOCAL INTERVENTION GEOMETRY

  • Under smooth local dynamics, responses factor into checkpoint-specific geometry and family-specific action coordinates, subject to explicit protocol conditions.The transfer chain requires a named pulse gauge, smooth local dynamics, spanning pulse excitation, positive source coverage, and bounded measurement and misspecification errors; cross-family coordinate equality remains a separate hypothesis.Source: §3.2 · pp. 4–5

3.3 IDENTIFYING THE CHECKPOINT RESPONSE OPERATOR

  • Target-independent pulses identify an action response when its coordinate lies in the pulse-coordinate span; full row rank identifies every action.The estimator uses four named standard pulses as coordinate anchors, with Q_m = I_4 in each family, and independent checkpoint copies for pulse branches.Source: §3.3 · p. 5

3.4 ESTIMATING A POOLED ACTION UNDER FAMILY HETEROGENEITY

  • Pooled action-coordinate recovery is bounded by source coverage and heterogeneity, so adding sources helps only when coverage gains outweigh the added errors.Under exact sharing, the heterogeneity term is zero; coverage alone does not establish sharedness.Source: §3.4 · pp. 5–6

3.5 PROSPECTIVE TRANSFER TO AN UNSEEN FAMILY

  • The operator predicts an unseen-family response by multiplying its pulse-estimated operator by a source-estimated pooled action coordinate.The end-to-end bound separates target identification, target locality, unseen-family coordinate bias, and source heterogeneity; direction and action recovery require the stated margin conditions.Source: §3.5 · p. 6

3.6 TWO PULSE READOUTS AND TASK-MATCHED ENDPOINTS

  • The direct readout fits a flexible source-only map from the pulse block, whereas the operator readout constrains predictions to the pulse-measured local geometry.The full L-State readout also uses current capability; the action-wise selector chooses between direct and operator readouts by development-family response-RMSE wins, with ties retaining the operator.Source: §3.6 · pp. 6–7
  • The retrospective shared-coordinate audit separately tests family and family–action deviations using observed family–action cells and a common-scale comparison.Its five-family retrospective mean is distinct from the prospective source-reference mean.Source: §3.6 · p. 7

4 THEORY-GUIDED EXPERIMENTAL DESIGN

The study compares pulse-based readouts in family-level development folds before evaluating frozen predictions on two sealed families. Disjoint data roles, a shared short-training protocol, and pre-opening prediction freezes structure the evaluation and its uncertainty analysis.

4 THEORY-GUIDED EXPERIMENTAL DESIGN

  • The design compares direct and operator pulse readouts in development, then tests structured transfer on a sealed family and an action-wise route on a second sealed family.The first sealed test also probes an action-specific calibration inversion, and repeat interventions examine its stability.Source: §4 · p. 7

4.1 DEVELOPMENT FAMILIES AND SEALED FAMILIES

  • Development uses 81 states across three model families; GLM-4-9B and Granite-3.1-8B each add 27 states, for 135 states across five families.Each family contributes three trajectories or seeds as described for its stage.Source: §4.1 · p. 7

4.2 DATA SEPARATION AND TRAINING CONTRACT

  • State generation, four synthetic standard pulses, semantic target training, and capability evaluation use disjoint task roles and exact normalized-text hashes.Primary branches use frozen base weights, rank-4 LoRA, eight AdamW steps, 32 examples, reset optimizer moments, and an identical checkpoint hash.Source: §4.2 · pp. 7–8

4.3 FAMILY-LEVEL GATING AND BASELINES

  • Development uses family-held-out folds and source-trajectory inner validation, while sealed predictions and utility-conditioned action choices are frozen before target-action training.Comparators include capability, direct and operator pulse readouts, scalar training statistics, a matched-state reference, and a target-specific early reference.Source: §4.3 · p. 8

4.4 METRICS AND UNCERTAINTY

  • Response error, direction, and utility decisions use distinct metrics, with bootstrap resampling and pulse permutations supporting uncertainty assessment.The sealed analyses resample whole trajectories; direction thresholds are source-frozen, and continuous response retains all scored components.Source: §4.4 · p. 8

4.5 RETROSPECTIVE FIVE-FAMILY SHAREDNESS AUDIT

  • The five-family sharedness audit opens only after sealed predictions are fixed and conditions its conclusions on five families, fixed pulses, and one training realization.Its primary comparisons use a common development-frozen response scale, complete trajectory blocks, and exact sign flips over 15 family–trajectory clusters.Source: §4.5 · p. 8

5 RESULTS

Pulse readouts improve on capability-only response prediction in development and on sealed GLM, while an action-wise selector performs strongly on sealed Granite. The five-family audit finds that operator coordinates vary across actions and families, and that modeling those deviations improves retrospective prediction.

5.1 L-STATE READOUTS IN THREE-FAMILY DEVELOPMENT

  • 39.4%: both pulse readouts reduce source-standardized MSE relative to capability alone across three-family development; direct has the lowest response RMSE, while operator has the highest sign balanced accuracy.The direct and operator RMSEs are 7.772 and 7.777, versus 9.987 for capability; full L-State reduces realized normalized regret from 0.472 to 0.248.Source: §5.1 · p. 9

5.2 SEALED GLM TEST OF THE OPERATOR READOUT

  • 78.3%: on sealed GLM, the operator reduces MSE relative to capability, while direct achieves a 71.8% reduction; operator sign balanced accuracy rises from 0.366 to 0.754.Full L-State lowers realized normalized regret from 0.592 to 0.434; the matched-state reference attains 0.410.Source: §5.2 · p. 9

5.3 FROM AN OPERATOR FAILURE TO ACTION-WISE READOUT SELECTION

  • The GLM science action has operator sign balanced accuracy zero while direct reaches 1.0 on determinate coordinates; repeat responses preserve the target sign in all 27 states, motivating action-wise selection.The frozen selector keeps operator for mathematics and reading and chooses direct for code and science; source-fit operator coordinates span −0.555 to 0.322, locating instability on the source-coordinate side without identifying a unique cause.Source: §5.3 · p. 9

5.4 SEALED GRANITE TEST OF ACTION-WISE READOUT SELECTION

  • 0.544: on sealed Granite, direct has the lowest single-readout RMSE; the selector reaches 0.554 versus 1.172 for capability.Operator aggregate sign balanced accuracy is 0.657 versus 0.654 for capability; full L-State lowers regret from 0.291 to 0.217 and raises top-1 from 53.1% to 63.6%.Source: §5.4 · p. 9

5.5 TESTING THE OPERATOR READOUT’S ACTION COORDINATE

  • The five-family audit finds action- and family-varying operator coordinates; a full family–action model reduces held-trajectory MSE by 17.3% overall and 48.0% for science.No action meets the retrospective one-dead-zone equivalence criterion; exact sharing is rejected for code, science, and reading, while mathematics remains unresolved by the difference test.Source: §5.5 · pp. 9–10

6 CONCLUSION

The conclusion presents L-State pulse readouts as complementary approaches to training-response prediction, with readout choice varying by action and endpoint. It also summarizes the sealed-family evaluations and the paper’s reproducibility and AI-use statements.

6 CONCLUSION

  • Pulse readouts improve response prediction across development and sealed families, while direct and structured readouts cover complementary transfer regimes.The conclusion also reports action-ranking gains from the full L-State readout and a five-family audit of operator-coordinate deviations.Source: §6 · p. 10

AI USE STATEMENT

  • The authors report using generative AI across conceptual, theoretical, experimental, and writing tasks, and say they reviewed and checked the assisted work.They take responsibility for the final content, including text, claims, code, and artifacts.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper links headline results to frozen evidence files and fresh-process replay receipts, with proofs, experimental details, tables, and audits in Appendices A–K.Sections 4–5 specify the data separation, training contract, frozen prediction protocol, metrics, and replay procedure.Source: REPRODUCIBILITY STATEMENT · p. 10

A SEALED-FAMILY SUMMARY TABLES

  • The sealed-family tables describe frozen GLM and Granite evaluations, including which readouts serve response, direction, and action prediction.The Granite action-wise selector is fitted on three development families; the table captions do not state metric values.Source: Table 2 · p. 13, Table 1 · p. 13

B CORE PROOFS

The core proofs characterize when capability can determine responses, derive an operator factorization under smooth local dynamics, and establish conditions for pulse identification and cross-family transfer. They also give bounds for source-coordinate estimation and response, sign, and action decisions.

B.1 CAPABILITY LEVEL SETS

  • Responses can be represented as a function of capability and action exactly when they are constant on every capability level set.The proof defines that function using the common response within each level set.Source: §B.1 · p. 13

B.2 FACTORIZATION FROM SMOOTH DYNAMICS

  • Under deterministic gradient flow with primitive losses and locally Lipschitz induced capability velocity, the operator columns recover capability-Jacobian-weighted loss gradients with an O(h∥a_m∥^2) remainder.The displayed column formula is B_:,j = −J_c(x)∇L_j(x).Source: §B.2 · p. 14

B.3 PULSE IDENTIFIABILITY AND DESIGN

  • A target action is identifiable from pulse responses when it lies in the pulse span; the stated minimax equality condition requires tight energy and equal singular values.The equality condition is QQ⊤ = (E0/r)I_r.Source: §B.3 · p. 14

B.4 SOURCE COVERAGE AND ACTION COORDINATES

  • The source-coordinate bounds require adequate source coverage, including τ_S < σ_S, and the ridge estimator has an explicit source bound under the stated rank or regularization conditions.The proof accounts for source operator error, pooling mismatch, and response error.Source: §B.4 · p. 15

B.5 UNSEEN-FAMILY TRANSFER AND DECISIONS

  • The unseen-family transfer bound includes target-pulse error, source-coordinate error, and target-family coordinate heterogeneity, with smooth-dynamics curvature able to scale as (A + D⋆,u)^2.The section also gives margin-based conditions and bounds for sign accuracy, action selection, and normalized regret.Source: §B.5 · pp. 15–16

C COMPLETE EXPERIMENTAL PROTOCOL

The experimental protocol branches standardized episodes from shared base models, evaluates paired capability changes, and freezes predictions before target labels are opened. It specifies training, response measurement, and leakage controls.

C COMPLETE EXPERIMENTAL PROTOCOL

  • The protocol uses the same ridge objective for direct pulse and full L-State readouts, while the structured operator readout uses a separate objective.The direct readout uses p(s) as features; the full-state readout uses [c(s); p(s)].Source: C COMPLETE EXPERIMENTAL PROTOCOL · p. 16

C.1 STATE GENERATION

  • Each trajectory branches from the same base model and repeats four synthetic generators in a balanced eight-episode order, producing 27 states per family.Three fixed trajectory seeds provide nine states per trajectory.Source: §C.1 · p. 16

C.2 PRIMARY TRAINING EPISODE

  • Training freezes the base model and uses a zero-output LoRA adapter with rank 4, eight optimizer steps, and 32 examples per episode.Each branch initializes new optimizer moments and restores the source adapter, checking that its tensor hash matches exactly.Source: §C.2 · p. 16

C.3 RESPONSE EVALUATION

  • Capabilities are mean negative completion-token NLLs clipped per token to [0, 20], and the primary response is the paired per-example difference.Before and after evaluations use the same 32 held-out examples and tokenizer outputs.Source: §C.3 · p. 16

C.4 LEAKAGE CONTROLS

  • Leakage controls use disjoint source indices, zero exact cross-role text intersections, low synthetic pulse-to-target 13-gram overlap, and a label vault opened after prediction serialization.The analysis reads validated completed manifests.Source: §C.4 · p. 16

D DEVELOPMENT RESULTS

The experiments compare pulse readouts across development and sealed families, then examine response slices, repeated interventions, readout selection, and coordinate heterogeneity. Results show endpoint-specific readout performance and a selector fitted on development-family wins.

D.1 ALIGNED-PULSE PERMUTATION

  • The aligned-pulse permutation analysis shuffles whole pulse-response matrices within source families, refits the source models, and scores unchanged target families.It reports one-sided fold p-values against preregistered null 95th percentiles.Source: §D.1 · p. 17

D.2 SOURCE-FIT BLEND WEIGHTS

  • The development-suite blend raises sign balanced accuracy from 0.589 to 0.643 and reduces RMSE from 8.293 to 7.876.The GLM seal uses the registered operator/full-state endpoint assignment rather than this blend.Source: §D.2 · p. 18

E.1 FREEZE ORDER AND COMPLETE COMPARISON

  • The prospective GLM comparison freezes 540 response predictions and 810 action choices before target labels open; its direction set has 169 positive and 15 negative responses.The general coordinate falls inside its source-frozen dead zone for all 108 records.Source: §E.1 · p. 18, Table 7 · p. 18, §E.2 · p. 18

E.3 ACTION AND TRAJECTORY SLICES

  • GLM operator MSE reduction ranges from 78.1% to 78.5% across its three prospective trajectories.The corresponding operator sign-BA range is [0.705, 0.860], and full-state regret reduction ranges from 25.0% to 29.9%.Source: Table 10 · p. 19, §E.3 · p. 19

F.1 FIVE-REPEAT SCIENCE INTERVENTION

  • Across 27 GLM states, the science-coordinate response has mean within-state standard deviation 0.00446 and unanimous five-repeat sign.The source science-action coordinate has norm 0.0539, while held-out-family fits range from 0.0588 to 1.108.Source: §F.1 · p. 19, Table 11 · p. 19

F.2 DEVELOPMENT-FITTED READOUT SELECTION

  • The selector assigns the operator readout to mathematics and reading and the direct readout to code and science after strict wins in at least two of three development folds.Ties retain the operator readout.Source: §F.2 · pp. 19–20, Table 12 · p. 20
  • On Granite, the action-wise pulse selector reaches pooled RMSE 0.5537 and action-macro RMSE 0.5479.The selector uses the direct readout for code and science and the operator readout for mathematics and reading.Source: §F.2 · pp. 19–20, Table 12 · p. 20

G.1 OPERATOR RECOVERY AND ESTIMATION CONTRACT

  • The five-family audit defines D_u as common-scale pooling error and S_u as the shared-signal fraction; ridge 0.7 applies only to coordinate-stability summaries.The tests and predictive comparisons use unpenalized least squares.Source: §G.1 · p. 20

G.2 DIFFERENCE, INTERACTION, AND PREDICTIVE DIAGNOSTICS

The five-family diagnostics find that family-aware modeling improves retrospective held-trajectory prediction over a shared model, while the observed family set constrains the scope of the audit.

G.2 DIFFERENCE, INTERACTION, AND PREDICTIVE DIAGNOSTICS

  • The joint restricted-residual sign-flip values are 6.10 × 10^-5 for family main effects, 1.83 × 10^-4 for total deviation, and 3.05 × 10^-4 for the interaction increment.These fixed-15-cluster diagnostics rely on approximate restricted-residual sign symmetry and are not tests over a population of model families.Source: §G.2 · p. 21
  • The full model improves descriptive held-trajectory MSE by 17.3% overall over the shared model, with gains of 0.3% for mathematics, 18.8% for code, 48.0% for science, and 1.8% for reading.The three folds have heavily overlapping training sets, so their resampled ranges are descriptive rather than formal confidence intervals.Source: §G.2 · p. 21, Table 14 · p. 21
  • Deleting one family at a time leaves the science and reading effects present after removing Mistral, but their magnitudes depend on the observed family set.The analysis is limited to five families, fixed pulse measurements, the chosen ridge, and one training realization.Source: §G.2 · p. 21

H.1 COMPLETE CORE COMPARISON

The Granite seal reproduces response and action gains, while identification checks and ablations characterize the pulse assay and local response geometry.

H.1 COMPLETE CORE COMPARISON

  • The sealed Granite result reproduces large response and action gains while replacing the GLM science-action inversion with non-degenerate direction estimates.The Granite comparison reports source-standardized RMSE and coordinate-macro direction balanced accuracy.Source: Table 15 · p. 22, §H.1 · p. 22

I.1 PULSE IDENTIFICATION AND LOCAL-DYNAMICS CHECKS

  • Pulse measurements have median intraclass correlation 0.967 and median signal-to-noise ratio 29.2, with 19 of 20 reliability cells at ICC 0.75 or above.This stability supports endpoint-specific readout selection within a reliable pulse assay.Source: §I.1 · p. 22
  • Four pulses yield the best reported RMSE at 8.293 and action regret at 0.248, while sign balanced accuracy peaks with two pulses at 0.615.The one-, two-, three-, and four-pulse RMSE values are 9.108, 9.614, 8.358, and 8.293, respectively.Source: §I.1 · p. 22
  • Controlled pulse mixtures have median relative residual 0.342, compared with 0.747 when exposure changes from eight steps to four or sixteen.A fixed-step amplitude scan has median relative residual 0.757 away from the reference amplitude.Source: §I.1 · p. 22
  • At four, eight, and sixteen steps, full L-State reduces source-standardized RMSE against capability by 45.3%, 37.3%, and 25.1%, respectively.The improvements hold across all nine duration-by-trajectory slices, but coordinate stability is measured within GLM only.Source: §I.1 · p. 23

I.2 REGISTERED DURATION EXPERIMENT

The registered duration experiment compares four-, eight-, and sixteen-step exposures on nine GLM states. Its results and artifact lineage document the experiment's scope and reproducibility.

I.2 REGISTERED DURATION EXPERIMENT

  • At four, eight, and sixteen steps, capability/full-L-State source-standardized RMSE is 1.2801/0.7000, 1.3147/0.8240, and 1.4574/1.0922.These correspond to reductions of 45.3%, 37.3%, and 25.1%, respectively.Source: §I.2 · p. 23, Table 18 · p. 23
  • Across all nine duration-by-trajectory slices, full L-State has lower response RMSE and lower action regret than capability.Duration-specific operator coordinates have cosine 0.952–0.997 at four steps and 0.973–0.992 at sixteen steps relative to eight steps.Source: §I.2 · p. 23

K REPRODUCIBILITY AND ARTIFACT LINEAGE

Loading 2609.08618v1…