Source-linked AI summary
Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training
Tingyun Li, Wenfeng Feng, Weiqing Li, Abudukelimu Wuerkaixi, Guohua Liu, Yuewei Zhang
TL;DR
Autonomous post-training needs to determine when evidence from an earlier update remains applicable after the parent model changes. The paper introduces BCIT, which binds evidence to source context, checks conditions, validates unresolved cases before full training, and applies a shared adoption rule. Across three capabilities on one 4B model, BCIT authorizes fewer harmful updates and achieves higher equal-budget final-model quality than evaluated alternatives.
Problem
The central question is whether evidence from a past update justifies reusing it for a changed parent, given effects depend on the parent, data, and training stage.
Method
BCIT binds observed effects to source contexts, checks prespecified conditions and hard conflicts, validates unresolved candidates on the current parent, and separates authorization from adoption.
Results
Across finance reasoning, text-to-SQL, and function calling, BCIT authorizes fewer harmful candidates while retaining beneficial ones and achieves higher equal-budget final-model quality than evaluated shared-model alternatives.
Takeaways & Limitations
The results support treating experience authorization as a distinct problem and allocating post-training compute conditionally rather than unconditionally.
Takeaways & Limitations
Evidence covers one 4B model, three target capabilities, one retention benchmark, human-specified boundaries, and complementary rather than independent studies.
Abstract
from arXiv · showhide
Large language models offer broad capabilities, but adapting them to evolving domains, tools, and requirements often entails repeated post-training. Autonomous systems automate parts of this process by proposing updates, training candidates, and using evaluation feedback to select subsequent proposals. As evidence accumulates, a central problem emerges: which past update evidence remains actionable after subsequent training has changed the parent model? An update's effect depends on its parent, data, and training stage. Treating past success as context-free permission can waste compute. If the resulting child is promoted, it can also degrade the subsequent training trajectory. We formulate this problem as conditional experience transfer and introduce Boundary-Calibrated Intervention Transfer (BCIT), a method that authorizes experience reuse before weight-changing training. BCIT binds an observed effect to its source context, checks applicability conditions, vetoes candidates with named hard conflicts, and obtains current-state evidence through a bounded training trial when needed. Fully trained candidates still face a shared adoption rule, and only observed events extend memory. On one 4B model adapted across finance reasoning, text-to-SQL, and function calling, candidate updates exhibit heterogeneous target and retention effects across the evaluated contexts. Under matched candidates, evidence, and compute, BCIT authorizes fewer harmful updates and attains higher equal-budget final-model quality than the evaluated alternatives. These results support treating experience authorization as a distinct problem in autonomous post-training.
Introduction
The paper frames experience reuse as a context-dependent control problem in autonomous post-training and introduces BCIT to decide when historical evidence warrants current training. BCIT separates authorization from adoption, using source-bound evidence, applicability checks, validation, and a shared promotion rule.
- Motivation: Past update success supports effects only under its source model and conditions, not unconditional permission to spend full-training compute on a changed parent.The candidate-level decision is whether source-context evidence justifies rejecting or executing an update for the current context.
- Method: BCIT records source context, evidence provenance, applicability conditions, and named conflicts for each historical update.New proposals specify updates, conditions, and provenance without fabricating a source context or outcome.
- Method: Authorization chooses rejection, bounded current-parent validation, or full training, while validation checkpoints are never promoted.A hard conflict rejects the candidate; unresolved evidence triggers a real budget-capped trial; satisfying the frozen rule authorizes training.
- Method: A separate shared adoption rule determines whether each fully trained child becomes the next parent, so authorization does not imply promotion.This separation distinguishes selecting a beneficial full run from promoting its trained child.
- Evaluation: The evaluation adapts one Qwen3-4B model across finance reasoning, text-to-SQL, and function calling while retaining instruction following.Sequential promotions create a stress test across parent checkpoints; the capabilities define the evaluation setting rather than BCIT’s objective.
- Related Work: Prior work addresses pipeline automation, transferability estimation, retrieval, or sequential adaptation, but not this candidate-level authorization decision.BCIT instead operates before full training or model integration and represents decisions using source evidence, applicability conditions, and conflicts.
Boundary-Calibrated Intervention Transfer
BCIT authorizes updates by combining source-context evidence with current applicability and hard-conflict checks before full training. It separates authorization, bounded validation, adoption, and memory updates so past evidence does not become context-free permission.
- Authorization: BCIT evaluates historical candidates using source strength, current compatibility, and a non-compensable hard-conflict veto.High source strength cannot compensate for low compatibility, and authorization occurs before weight-changing training.
- Authorization: Applicability conditions are frozen before target outcomes open and cover parent stage, data composition, optimization regime, output interface, and evaluation contract.Missing information is unresolved, while a named contradicted condition constitutes a hard conflict.
- Authorization: A candidate is rejected, validated, or fully trained according to conflict status, transparent score thresholds, evidence grade, and faithful validation availability.BCIT uses τl = .30 and τh = .70; direct training additionally requires grade A evidence, while unavailable faithful validation causes rejection.
- Authorization: New proposals receive no source score and must pass executability, hard-conflict, and faithful-short-run checks before validation.They cannot bypass validation through provenance or an artificial source score.
- Current-State Validation: Validation runs from the current parent, and Pass, Fail, or Inconclusive outcomes respectively route candidates to full training, rejection, or a budget-gated fallback.Validation checkpoints are temporary and are never promoted directly.
- Adoption: Every fully trained candidate faces the same fixed promote-or-rollback adoption rule using disjoint adoption data and target-retention constraints.The rule is shared across authorization policies, so authorization does not imply promotion.
- Memory and Exploration: Within an episode, the curated library is immutable; only executed validations and full runs create observed events, while rejections create no effect label.Observed failures remain bound to their parent, and later library versions require provenance checks, deduplication, and conflict review.
- Memory and Exploration: A frozen knowledge snapshot may generate at most q = 3 atomic proposals when a prespecified coverage gap or stall occurs.The proposal generator cannot alter evaluators or rewards, access sealed outcomes, or bundle uncontrolled interventions.
Experiments and Results
The experiments test effect heterogeneity, authorization decisions, bounded validation, and complete-policy performance under matched evidence and compute. BCIT reduces harmful authorization while preserving beneficial coverage and achieves stronger equal-budget outcomes than evaluated alternatives.
- RQ1: Effect Heterogeneity: 13 of 24 candidates do not improve their target, and only 3 of 11 target-improving candidates also improve both retention measures.A supervised SQL update gains 2.25 target points but loses 22.74/18.71 IFEval-P/I points; a function-calling update gains 1.35 but loses 11.09/8.15.
- RQ2: Authorization: BCIT authorizes 2 of 8 harmful and 9 of 10 beneficial candidates, compared with 5 and 8 for Flat-Additive.Harmful authorization is 25.0% versus 62.5%, while beneficial-candidate coverage is 90.0% versus 80.0%.
- RQ3: Current-Parent Validation: 20 of 24 candidates have matching short- and full-training target directions, with Spearman ρ = .72 under the nominal 20% validation budget.The three-way screen yields 10 Pass, 9 Fail, and 5 Inconclusive decisions; Pass has 80.0% precision and 80.0% recall at a median 17% of full-training cost.
- RQ4: Equal-Budget Outcomes: BCIT improves the final mean over Flat-Additive in all six paired runs, with a mean gain of 2.63 points.The 95% CI is [2.10, 3.16] and the exact sign-flip p-value is .03125.
- RQ4: Equal-Budget Outcomes: BCIT exceeds Validate-All by 1.50 points and Additive+Veto by 0.90 points, while using 0.74 fewer GPU-hours than Validate-All.Each difference is positive in all six pairs, with p = .03125.
- RQ4: Trajectories: BCIT’s AUC is 44.9 versus 43.6 for Flat-Additive and 44.2 for Validate-All before the endpoint.The paired BCIT–Validate-All gain is 0.70 [0.22, 1.18], while the 0.47-point gap to Additive+Veto includes zero.
Limitations
The evidence is bounded to one 4B model, three target capabilities, one retention benchmark, and human-specified boundaries. Broader domains, learned boundaries, and longer trajectories remain future work.
- Scope: The evidence covers one 4B model, three target capabilities, one retention benchmark, and human-specified boundaries.The four studies are complementary rather than independent replications, and component comparisons remain descriptive.
- Open Boundaries: Validation fidelity and compute trade-offs may vary with model scale, data, and schedule.The paper identifies learned boundaries, broader domains, and longer trajectories as future work.
Conclusion
The paper treats experience authorization as distinct from training and adoption decisions. BCIT binds evidence to source conditions, validates unresolved candidates on the current parent, and applies a shared promote-or-rollback rule.
- Conclusion: BCIT rejects named hard conflicts, validates unresolved candidates on the current parent, and grants full-training budget only under a frozen rule.A shared adoption rule then promotes or rolls back each trained child.
- Conclusion: Across the evaluated evidence chain, update effects vary across contexts, while BCIT authorizes fewer harmful updates under matched information.
Supplementary Material
Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training.
- Title: The paper title names conditional experience transfer as the focus of autonomous LLM post-training.
Scope and Study Separation
The supplement separates four empirical components and specifies frozen, evidence-bound authorization and adoption rules for candidate updates. These components address distinct questions without pooling their observations into a single effect.
- The four empirical components remain separate: Retrospective-24 diagnoses heterogeneous effects, Audit-24 evaluates frozen authorization decisions, ShortFull-24 tests validation, and end-to-end episodes assess sequential policies.
- BCIT records source context, evidence, applicability conditions, and conflicts before routing candidates to Reject, Validate, or Train.
- Missing metadata is unresolved rather than vetoing a candidate, while a hard conflict requires a named contradiction in a required applicability field.
- A new proposal must validate when no hard conflict exists and a faithful short-run compilation is available; exploration is capped at three atomic proposals per episode.
- A trained child is promoted only after target, non-primary capability, utility, retention, and execution conditions pass; otherwise the parent is restored.
- Validate-All ignores source strength, applicability, and conflicts after shared executability checks, whereas Flat-Additive treats conflicts as ordinary additive features.
Data, Training, and Evaluation
The study adapts one model across finance, SQL, and function-calling settings while using disjoint data roles and deterministic evaluation. Instruction following serves as a retention guardrail rather than a compensable target.
- FinQA, Spider, and xLAM provide source experience; TAT-QA, BIRD, and BFCL evaluate related target contexts; IFEval measures instruction-following retention.
- IFEval has 541 prompts and 834 verifiable instructions, with prompt-level and instruction-level strict accuracy reported separately.
- Target improvement is required, while IFEval is a non-compensable feasibility condition rather than an averaged benefit term.
- Candidate cards change one declared primary factor, and full-run lengths remain candidate-specific rather than sharing an artificial step count.
- Evaluation uses one deterministic sample per prompt and official parsers, database execution, structured-call checking, and strict instruction evaluation; unusable outputs are hard failures.
- The target-context full outcome is excluded from the controller-visible historical record at authorization time.
RQ1: Retrospective Transfer Diagnostic
Retrospective candidate effects are heterogeneous across finance, SQL, and function calling, and target gains often conflict with retention. Thus, prior success does not reliably transfer across contexts.
- 11/24 target effects are positive, 3/24 are neutral, and 10/24 are negative across candidate–target-context pairs.
- Only 3 of 11 target-improving updates also improve both retention measures.
- Every capability family contains both increases and decreases, so semantic relatedness does not guarantee positive transfer.
- Standard SQL SFT improves BIRD by 2.250 pp while reducing IFEval-P/I by 22.736/18.705 pp.
- Random-sampling function SFT improves BFCL by 1.351 pp while reducing IFEval-P/I by 11.091/8.153 pp.
RQ2: Outcome-Blind Authorization Audit
The outcome-blind audit scores authorization decisions against sealed candidate outcomes using shared count-based metrics. BCIT’s paired comparison is directionally favorable but not statistically precise in this cohort.
- The endpoint-table policies share seed-indexed candidate streams and adoption rule g, while remaining baselines use their stated action spaces descriptively.
- Audit-24 contains 10 Beneficial, 8 Harmful, and 6 Neutral units, with decisions made from frozen decision-time evidence before full-run labels open.
- Audit policies report Coverage, harmful-event rate, beneficial coverage, utility, and Harm/Auth. from authorized outcome counts.
- The paired discordance table shows harmful units as both 2, Flat-only 3, BCIT-only 0, and neither 3.
- The exact two-sided McNemar test gives p = .25, so the harmful-decision comparison is directional cohort evidence rather than a precise population error-rate estimate.
RQ3: Bounded-Validation Fidelity
At the primary 20% validation budget, bounded trials provide useful but imperfect evidence: short and full target directions agree for most candidates, while some sign reversals remain. BCIT’s matched-compute endpoint advantage persists against evaluated alternatives, with process and ablation results clarifying the trade-offs.
- Bounded-validation fidelity: 83.3% of candidates had matching short- and full-run target directions at the primary 20% validation budget.Agreement was 20/24 candidates, with Spearman ρ = .720.
- Bounded-validation fidelity: Four sign reversals and two false Pass cases show that bounded validation supplies current-parent evidence but does not replace full training or shared adoption.The sign reversals were S04, S07, G04, and G10.
- Matched-compute endpoint: BCIT exceeded Validate-All by 1.496 ± .614 pp and Additive+Veto by .900 ± .596 pp in paired cross-task mean.Each comparison had six positive differences and exact two-sided sign-flip p = .03125; BCIT used .740 fewer GPU-hours than Validate-All.
- Component ablations: Removing the hard veto lowered the mean by 1.487 ± .679 pp, while rejecting every validation-routed candidate lowered it by 1.764 ± .617 pp.These three-seed component results were descriptive and received no significance test.
- Process behavior: BCIT promoted 46 of 62 fully trained candidates, compared with 28 of 63 for Flat-Additive, yielding 74.2% versus 44.4%.These counts used the shared adoption rule and were reported as process descriptors rather than independent statistical units.
Statistical Calculations and Claim Map
The paper’s inferential claims are tied to specific sampling units, paired tests, and evidence tables. Its claim-to-evidence map separates transfer, authorization, validation, endpoint, and process conclusions while limiting interpretation of cohort-level intervals.
- Statistical calculations: End-to-end inference uses six complete episodes as sampling units, with paired intervals based on five degrees of freedom.The exact sign-flip test enumerates 64 sign assignments and has a smallest attainable two-sided value of .03125.
- Statistical calculations: Audit-24 authorization rates use two-sided exact binomial intervals, while harmful-action comparisons use a paired 2×2 discordance table and exact McNemar testing.These intervals describe the evaluated cohort and are not cluster-robust population intervals.
- Claim-to-evidence map: The claim-to-evidence map assigns transfer heterogeneity to Table 9, authorization to Tables 10–11, validation fidelity to Figure 4 and Tables 12–13, endpoints to Tables 14–17, and process behavior to pooled promotion and rollback counts.These links locate the candidate- or episode-level evidence supporting the corresponding main-paper claims.