PaperlayerView exact revision on arXiv

Source-linked AI summary

Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training

Tingyun Li, Wenfeng Feng, Weiqing Li, Abudukelimu Wuerkaixi, Guohua Liu, Yuewei Zhang

arXiv:2608.26730v1cs.AI

TL;DR

After training changes a model’s parent, it is unclear whether earlier update evidence still warrants reuse. BCIT binds evidence to its source context and validates unresolved candidates on the current parent; under equal compute, it exceeds Validate-All by 1.50 points and Additive+Veto by 0.90 points.

  • Problem

    The open question is whether evidence that an update helped in its source context still justifies full-training compute after later training changes the parent.

    Source: Introduction · p. 1

  • Method

    BCIT binds observed effects to source conditions, checks applicability, vetoes named hard conflicts, validates unresolved candidates on the current parent, and applies a shared adoption rule.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Results

    BCIT exceeds Validate-All by 1.50 points and Additive+Veto by 0.90 points in equal-budget endpoint quality, with each difference positive in all six paired seeds.

    Source: Does the Complete Policy Improve… · p. 7

  • Takeaways & Limitations

    The results support evidence-conditioned rather than unconditional allocation of post-training compute.

    Source: Conclusion · p. 7

    Evidence covers one 4B model, three target capabilities, one retention benchmark, and human-specified boundaries.

    Source: Limitations · p. 7

Abstract

from arXiv · show

Large language models offer broad capabilities, but adapting them to evolving domains, tools, and requirements often entails repeated post-training. Autonomous systems automate parts of this process by proposing updates, training candidates, and using evaluation feedback to select subsequent proposals. As evidence accumulates, a central problem emerges: which past update evidence remains actionable after subsequent training has changed the parent model? An update's effect depends on its parent, data, and training stage. Treating past success as context-free permission can waste compute. If the resulting child is promoted, it can also degrade the subsequent training trajectory. We formulate this problem as conditional experience transfer and introduce Boundary-Calibrated Intervention Transfer (BCIT), a method that authorizes experience reuse before weight-changing training. BCIT binds an observed effect to its source context, checks applicability conditions, vetoes candidates with named hard conflicts, and obtains current-state evidence through a bounded training trial when needed. Fully trained candidates still face a shared adoption rule, and only observed events extend memory. On one 4B model adapted across finance reasoning, text-to-SQL, and function calling, candidate updates exhibit heterogeneous target and retention effects across the evaluated contexts. Under matched candidates, evidence, and compute, BCIT authorizes fewer harmful updates and attains higher equal-budget final-model quality than the evaluated alternatives. These results support treating experience authorization as a distinct problem in autonomous post-training.

Introduction · Related Work · Problem Formulation

The paper frames experience reuse as a context-dependent decision: past update effects remain bound to their source conditions as training changes the parent. BCIT uses that evidence to decide whether to reject, validate, or fully train a candidate, while separating authorization from promotion.

Introduction

  • BCIT rejects candidates with hard conflicts, validates when evidence is unresolved, and authorizes full training only when a frozen rule is satisfied; every trained child faces a shared adoption rule.Only executed events extend memory, and proposals without observed source effects must pass validation before full training.Source: Introduction · p. 1, Authorization, Adoption, and Budget · p. 3
  • The evaluation uses sequential promotions of one Qwen3-4B model across finance reasoning, text-to-SQL, and function calling, with instruction following as a retention constraint.It tests whether update effects vary across contexts, BCIT can reject harmful candidates while retaining beneficial ones, bounded validation is informative but imperfect, and the complete policy improves equal-compute final-model quality.Source: Introduction · pp. 1–2
  • The paper formulates conditional experience transfer as a control problem because each promoted child becomes the parent for later decisions, making source-bound evidence relevant to subsequent training.An observed gain supports an update only under its source model and conditions, not as unconditional permission to spend full-training compute from a changed parent.Source: Introduction · pp. 1–2

Related Work

State-Bound Decision Unit

  • The decision unit pairs one candidate with one current context; each candidate changes one declared data or training factor through a reproducible configuration difference from a fixed parent.Historical records bind the observed effect to its source context, evidence, applicability conditions, conflicts, and artifact provenance; new proposals specify conditions and provenance but have no fabricated source outcome.Source: State-Bound Decision Unit · p. 2

Authorization, Adoption, and Budget

  • Authorization allocates compute but does not itself replace the parent: only the shared adoption rule determines whether a fully trained child becomes the next parent.Before each evaluation episode, the library, candidate generator, data roles, thresholds, fallback, evaluators, adoption rule, and budget are frozen, and final test outcomes remain sealed until the episode ends.Source: Authorization, Adoption, and Budget · p. 3

Boundary-Calibrated Intervention Transfer

BCIT authorizes experience reuse before weight-changing training by conditioning historical evidence on the current context and vetoing hard conflicts. New proposals must validate, and every fully trained child faces the same adoption rule.

Boundary-Calibrated Intervention Transfer

  • BCIT places authorization between candidate generation and full training, routing historical candidates to rejection, bounded validation, or training after an executability check.New proposals cannot bypass validation, and validation checkpoints are temporary rather than promotable.Source: Boundary-Calibrated Intervention… · p. 3, Figure 1 · p. 4

Evidence and Applicability

  • BCIT combines frozen source strength with current compatibility, but a named hard conflict vetoes reuse and strong source evidence cannot offset poor compatibility.Only replicated matched-control evidence receives grade A and may skip validation; point estimates alone cannot bypass it. Applicability fields are frozen before target outcomes open.Source: Evidence and Applicability · p. 3

Authorization

  • Historical candidates are rejected below τl = .30 or on hard conflict, trained directly at or above τh = .70 only with eligible evidence, and otherwise validated when feasible.Thresholds are calibrated on units disjoint from the primary audit and frozen before its outcomes open; candidates without a faithful short run are rejected if routed to validation.Source: Authorization · p. 3

Current-State Validation

  • Validation runs from the current parent under a capped budget and passes only when target, retention, and execution requirements hold.A failure rejects the candidate and never promotes its temporary checkpoint; an inconclusive result proceeds only with a frozen full-run lease and sufficient remaining budget.Source: Current-State Validation · p. 4

Shared Adoption

  • Every fully trained candidate is promoted only if it meets the target criterion, satisfies all retention constraints, and avoids hard failure; otherwise the parent is retained.The adoption rule uses data disjoint from validation and is fixed and shared across authorization policies.Source: Shared Adoption · p. 4

Memory and Exploration

  • Episode libraries remain immutable, while histories record decisions and observed outcomes; rejections create no effect labels, and observed failures remain bound to their parent.After an episode, new library versions require provenance checks, deduplication, and conflict review; exploration may generate at most q = 3 atomic proposals, which must re-enter common checks and validate.Source: Memory and Exploration · pp. 4–5

Controlled Authorization Comparators

  • Comparators isolate authorization choices: Flat-Additive permits evidence to compensate for mismatch, Additive+Veto preserves the veto, and Validate-All validates every executable candidate.BCIT ablations remove either the hard veto or validation of candidates routed to that action; end-to-end contrasts use complete equal-budget policies because authorization changes later parents and opportunities.Source: Controlled Authorization Comparators · p. 5

Experiments and Results

Across matched candidate–context pairs, update effects varied sharply, while BCIT authorized fewer harmful candidates and retained beneficial ones in Audit-24. Under matched compute, BCIT achieved higher final mean performance than Flat-Additive and the prespecified alternatives.

Questions and Protocol

  • The evaluation uses six paired end-to-end seeds and a 36-GPU-hour cap for BCIT and Flat-Additive, with the same start, candidate stream, data order, and adoption rule.Audit-24 contains 10 beneficial, 8 harmful, and 6 neutral outcomes; decisions are frozen before full outcomes are opened.Source: Questions and Protocol · p. 5

How Stable Are Update Effects Across Evaluated Contexts? (RQ1)

  • 13 of 24 candidates failed to improve their target, and only 3 of 11 target-improvers also improved both retention measures.Examples include a supervised SQL update with a 2.25-point target gain but −22.74/−18.71 IFEval-P/I changes, and a random-sampling function-calling update with a 1.35-point gain but −11.09/−8.15 retention changes.Source: How Stable Are Update Effects Across… · p. 5, Figure 2 · p. 6

Does BCIT Reduce Harmful Authorization while Retaining Beneficial Candidates? (RQ2)

  • 25.0% versus 62.5%: BCIT authorizes fewer harmful candidates than Flat-Additive while covering 90.0% versus 80.0% of beneficial candidates.The harmful-candidate authorization comparison is directional cohort evidence (exact McNemar p = .25); removing applicability conditions raises harmful authorization to 75.0%, while adding a veto to the additive score lowers it to 37.5%.Source: Does BCIT Reduce Harmful Authorization… · pp. 5–6, Table 1 · p. 6

Does Bounded Current-Parent Validation Help? (RQ3)

  • 83.3%: short- and full-training target directions agree for 20 of 24 candidates, and Pass identifies beneficial full-run outcomes with 80.0% precision and recall.Pass is obtained at a median 17% of full-training cost; four sign reversals and two false Pass cases remain, so bounded validation provides current-parent evidence rather than replacing full training or shared adoption.Source: Does Bounded Current-Parent Validation… · p. 6

Does the Complete Policy Improve Equal-Budget Outcomes? (RQ4)

  • 2.63 points: BCIT exceeds Flat-Additive in all six paired runs, and beats Validate-All by 1.50 and Additive+Veto by 0.90 points.The 95% CI for the Flat-Additive gain is [2.10, 3.16] (exact sign-flip p = .03125); BCIT uses 0.74 fewer GPU-hours than Validate-All.Source: Does the Complete Policy Improve… · pp. 6–7, Table 2 · p. 7
  • 44.9 versus 43.6: BCIT’s budget-normalized AUC exceeds Flat-Additive’s, while its AUC advantage over Additive+Veto includes zero.BCIT’s paired gain over Validate-All is 0.70 [0.22, 1.18], but the 0.47-point gap to Additive+Veto includes zero. Three-seed component results are descriptive, and specialists exceed BCIT on their named task but use three budgets.Source: Does the Complete Policy Improve… · p. 7

Limitations

The evidence is narrow in model, task, benchmark, and boundary scope, and its studies are complementary rather than independent replications.

Limitations

  • Evidence covers one 4B model, three target capabilities, one retention benchmark, and human-specified boundaries; broader scales, domains, and longer trajectories remain untested.Only complete-policy comparisons use six paired seeds; component results are descriptive. Validation fidelity and compute trade-offs may vary with model scale, data, and schedule, while learned boundaries remain future work.Source: Limitations · p. 7

Conclusion

BCIT conditions experience reuse on current-parent evidence and, under matched compute, achieves a higher cross-task mean than evaluated shared-model alternatives.

Conclusion

  • Under matched information, BCIT authorizes fewer harmful candidates while retaining beneficial ones, and under matched compute it attains a higher cross-task mean than all evaluated shared-model alternatives.Update effects vary across contexts in the evaluated evidence chain.Source: Conclusion · p. 7
  • BCIT rejects named hard conflicts, validates unresolved candidates on the current parent, and grants full-training budget only under a frozen rule before shared-rule promotion or rollback.Past success in one source context does not itself authorize modifying every future parent.Source: Conclusion · p. 7

Supplementary Material · Scope and Study Separation · Controller and Episode Contract

The supplement keeps four empirical studies separate and specifies a frozen, conditional controller for end-to-end episodes. Its authorization rules distinguish evidence strength and applicability from validation and adoption.

Scope and Study Separation

  • Retrospective-24, Audit-24, ShortFull-24, and end-to-end episodes answer distinct questions; no pooled effect is computed across them.Retrospective-24 describes heterogeneous effects but does not establish policy superiority; Audit-24 freezes decisions, ShortFull-24 shares full outcomes with Audit-24, and end-to-end episodes model sequential policies that change later parents and opportunities.Source: Scope and Study Separation · p. 10

Controller and Episode Contract

  • Each end-to-end episode freezes its manifest and analysis specification, and seals final outcomes until the adopted checkpoint and GPU-hour ledger are fixed.The manifest includes the model, seed, memory and knowledge snapshots, candidate inventory, retrieval order, thresholds, evaluators, data roles, resource cap, stopping rule, and other controller settings.Source: Controller and Episode Contract · p. 10

Evidence Strength and Applicability

  • Only grade A replicated matched-control evidence may bypass current-state validation; grade B is a single matched internal comparison, and grade C is a provenance-checked external result or recipe.Unscored anecdotes receive no grade and cannot authorize training.Source: Evidence Strength and Applicability · p. 10
  • The source effect uses the lower confidence bound for replicated matched runs or the recorded point estimate otherwise, and negative effects are clipped to zero.The task MCID is one percentage point, the source reference improvement is twice that MCID, and κi = .02 raw score units.Source: Evidence Strength and Applicability · p. 10
  • Applicability averages five pre-training compatibility fields, while a hard veto requires a named contradiction; missing metadata remains unresolved rather than triggering a veto.The fields cover capability family, training data, runtime, current parent, and evaluator/output protocol; each receives 0, .5, or 1 for mismatch, unresolved, or match.Source: Evidence Strength and Applicability · p. 10

Authorization and Validation Rule

  • Historical candidates are rejected for a hard conflict or low authorization score, trained at a high score when indicated, and otherwise sent to validation when available.New proposals must validate absent a hard conflict when a faithful short-run compilation exists, with at most three atomic proposals per episode; proposals cannot alter evaluators, rewards, data roles, or sealed outcomes.Source: Authorization and Validation Rule · p. 10
  • Validation passes only when target, retention, and execution conditions hold; failures reject without promoting the temporary checkpoint, while inconclusive results require a reserved, affordable full-run lease.Pass routes to full training, Fail rejects in the current context, and Inconclusive otherwise leaves the parent unchanged; every fully trained candidate then faces the shared adoption rule on data disjoint from validation.Source: Authorization and Validation Rule · p. 10
  • All authorization policies share adoption thresholds: at least +0.5 pp primary-target gain, no non-primary capability below −1 pp, positive cross-capability utility, retention compliance, and no hard execution failure.Otherwise the parent is restored; the episode-level IFEval guardrail is −2 pp on both prompt- and instruction-level strict accuracy. Flat-Additive treats conflict additively, Additive+Veto retains hard rejection, and Validate-All validates every executable candidate; downstream rules are shared.Source: Authorization and Validation Rule · pp. 10–11

Data, Training, and Evaluation

The section assigns separate roles to source, transfer-target, and retention benchmarks, and specifies how training, authorization records, and final evaluation are kept distinct. It also defines candidate-specific training schedules and deterministic endpoint evaluation with hard-failure handling.

Dataset Roles

  • FinQA, Spider, and xLAM supply source-update experience; TAT-QA, BIRD, and BFCL measure related-context transfer; IFEval guards retention, with records disjoint by role.The benchmarks may share public origins, but training, validation, adoption, and final-evaluation records remain role-disjoint.Source: Dataset Roles · p. 11
  • IFEval’s 541 prompts and 834 verifiable instructions are scored separately at prompt and instruction levels, and retention is a non-compensable feasibility condition.Target improvement is required; IFEval is not averaged with target performance to make an update appear beneficial.Source: Dataset Roles · p. 11

Common Training Defaults

  • Candidate cards vary one declared primary factor, inherit Table 7 otherwise, and retain candidate-specific full-run lengths rather than sharing an artificial step count.Short continuations and longer schedules follow their execution plans.Source: Common Training Defaults · p. 11
  • Endpoint evaluation uses one temperature-zero sample per prompt, task-specific official-style evaluators, and treats API errors, missing predictions, incompatible protocols, or unusable checkpoints as hard failures.Finance uses a frozen numerical parser, SQL executes against its declared backend, BFCL uses its structured-call checker, and IFEval uses its strict evaluator.Source: Common Training Defaults · p. 11

Intervention Record Example

RQ1: Retrospective Transfer Diagnostic

Matched retrospective pairs show mixed target effects across finance, SQL, and function calling; most target-improving updates did not also improve both retention measures.

RQ1: Retrospective Transfer Diagnostic

  • Across 24 pairs, 11/24 target effects were positive, 3/24 neutral, and 10/24 negative; only 3 of 11 target-improving updates improved both retention measures.Each control/intervention pair used the same parent, evaluation examples, data scale, and budget.Source: RQ1: Retrospective Transfer Diagnostic · p. 11
  • Finance had five target increases and three decreases; SQL had three increases, two neutral outcomes, and three decreases; Function Calling had three increases, one neutral outcome, and four decreases.Source: RQ1: Retrospective Transfer Diagnostic · p. 11
  • Standard SQL SFT improved BIRD by 2.250 pp while reducing IFEval-P/I by 22.736/18.705 pp.This illustrates the risk of selecting updates on target performance alone.Source: RQ1: Retrospective Transfer Diagnostic · p. 11
  • Random-sampling function SFT improved BFCL by 1.351 pp while reducing IFEval-P/I by 11.091/8.153 pp.This is a second example of target-only selection harming retention.Source: RQ1: Retrospective Transfer Diagnostic · p. 11

RQ2: Outcome-Blind Authorization Audit

The outcome-blind audit compares BCIT and Flat-Additive using decisions frozen from decision-time evidence, then opens full-run labels only for scoring. BCIT authorized fewer harmful units in this cohort, though the paired evidence is directional rather than a precise population estimate.

RQ2: Outcome-Blind Authorization Audit

  • Audit-24 contains 10 Beneficial, 8 Harmful, and 6 Neutral units; decisions use frozen decision-time evidence, with full-run labels opened only for scoring.Beneficial units meet the target MCID and retention constraints without hard failure; Harmful units degrade the target, violate retention bounds, or incur hard failure; other units are Neutral.Source: RQ2: Outcome-Blind Authorization Audit · pp. 11–12
  • The audit reports Coverage = (Bπ + Hπ + Nπ)/24, HER = Hπ/8, BCov = Bπ/10, SelU2 = (Bπ − 2Hπ)/24, and Harm/Auth. = Hπ/(Bπ + Hπ + Nπ).These count-based metrics define the comparison without imposing a posthoc ordering over tied decisions.Source: RQ2: Outcome-Blind Authorization Audit · p. 12
  • On harmful units, both policies authorized 2, Flat-Additive alone authorized 3, BCIT alone authorized 0, and neither authorized 3; exact two-sided McNemar p = .25.The passage characterizes this as directional cohort evidence, not a precise population error-rate estimate.Source: RQ2: Outcome-Blind Authorization Audit · p. 12

RQ3: Bounded-Validation Fidelity · RQ4: Matched 36-GPU-Hour Episodes

In matched 36-GPU-hour episodes, BCIT improves the shared model’s cross-task result over the matched alternatives, while bounded validation predicts full-training target directions for most candidates but still leaves reversals and false passes. The results also show that hard vetoes and promotion decisions matter to the final trajectory.

RQ3: Bounded-Validation Fidelity

  • At the 20% validation budget, short and full target directions agreed for 20/24 candidates (83.3%; 95% interval [62.6%, 95.3%]), with Spearman ρ = .720.SFT and GRPO each had 10/12 sign agreements; pass versus non-pass yielded 80.0% precision, 80.0% beneficial recall, and a 14.3% false-positive rate. Four sign reversals and two false Pass cases remained, so bounded validation did not replace full training or the shared adoption rule.Source: RQ3: Bounded-Validation Fidelity · pp. 12–13, Table 12 · p. 14, Figure 4 · p. 14

RQ4: Matched 36-GPU-Hour Episodes

  • BCIT, Flat-Additive, Validate-All, and Additive+Veto shared six seed-indexed starts, candidate streams, a common adoption rule, and a 36-GPU-hour cap.BCIT–Validate-All and BCIT–Additive+Veto were prespecified comparison order; paired candidates also shared data order. The anytime grid ran from 0 to 36 GPU-hours in six-hour increments, carrying the final adopted parent forward if an episode stopped early.Source: RQ4: Matched 36-GPU-Hour Episodes · pp. 13–14

Primary Paired Comparison

  • Against Flat-Additive, BCIT’s paired cross-task mean advantage was 2.630 ± .506 pp (95% paired t interval [2.099, 3.161]); all six differences were positive (exact sign-flip p = .03125).Task-level paired differences were +2.483 pp for TAT-QA, +3.042 pp for BIRD, and +2.365 pp for BFCL.Source: Primary Paired Comparison · p. 14, Table 14 · p. 15

Protocol-Amendment Baselines

  • BCIT exceeded Validate-All by 1.496 ± .614 pp and Additive+Veto by .900 ± .596 pp; both comparisons had six positive differences and exact two-sided sign-flip p = .03125.Against Validate-All, BCIT used .740 fewer GPU-hours on average and improved AUC by .702; the .466 AUC gap to Additive+Veto was treated as a secondary trend, not a separate superiority result.Source: Protocol-Amendment Baselines · p. 14, Table 15 · p. 15

Component Ablations

  • Removing the hard veto lowered the mean by 1.487 ± .679 pp relative to BCIT on S1–S3; rejecting every validation-routed candidate lowered it by 1.764 ± .617 pp.The reject-unresolved ablation used less compute because it declined unresolved candidates, which was not an equal-quality efficiency result.Source: Component Ablations · p. 14, Table 16 · p. 15

Endpoint and Process Summaries

  • Across six paired episodes, BCIT promoted 46/62 fully trained candidates (74.2%) and rolled back 16, versus Flat-Additive’s 28/63 promotions (44.4%) and 35 rollbacks.These are process descriptors under the shared adoption rule, not independent statistical units; specialist models were concentration references, not shared-model baselines.Source: Endpoint and Process Summaries · p. 14
  • An earlier 24-GPU-hour pilot ended with BCIT, BCIT-Retrieve, and Flat-Additive rolled back to Base (cross-task mean 42.514; IFEval-P 76.525), and was not substituted for the finalized study.Static Multitask retained a degraded child with cross-task mean 25.600 and IFEval-P 55.638.Source: Endpoint and Process Summaries · p. 15

Statistical Calculations and Claim Map

The section specifies how uncertainty is calculated and which evidence supports the paper’s claims. End-to-end inference treats six complete episodes as the sampling units, while audit intervals describe only the evaluated cohort.

Statistical Calculations and Claim Map

  • Six-seed means and sample standard deviations are recomputed from seed rows; paired task, retention, and cross-task intervals use two-sided Student t intervals with five degrees of freedom.The exact sign-flip test enumerates 64 assignments, assumes exchangeable paired-difference signs under the sharp null, and has a minimum attainable two-sided value of .03125 with six pairs; complete episodes—not checkpoints, candidates, tasks, or evaluation examples—are the end-to-end sampling units.Source: Statistical Calculations and Claim Map · p. 15
  • Audit-24 marginal rate intervals are exact two-sided binomial intervals, while harmful-action comparisons use paired 2×2 discordance tables and an exact two-sided McNemar test.Because audit units may share capability or recipe structure, these intervals describe the evaluated cohort rather than cluster-robust population intervals.Source: Statistical Calculations and Claim Map · p. 15
  • Table 17 reports all 36-GPU-hour endpoints, with task and retention means and cross-task Mean and GPUh reported with sample standard deviations.Secondary baselines are descriptive aggregates and are not used for inferential claims.Source: Table 17 · p. 16
  • The claim map links transfer heterogeneity to Table 9, authorization quality to Tables 10–11, validation fidelity to Figure 4 and Tables 12–13, and equal-budget endpoints to Tables 14–17.Process behavior is linked to pooled promotion and rollback counts; statistical calculations use unrounded table values and accompanying CSV files where provided.Source: Statistical Calculations and Claim Map · p. 16

Related papers

Loading 2608.26730v1…