PaperlayerView exact revision on arXiv

Source-linked AI summary

When to Forget: Risk-Controlled Context Adaptation for Long-Horizon Agents

Chenle Chen, Yangbo Wei, Zhijie Zheng, Zhenyu Su, Qifan Wang, Junhong Qian, Chao Yao, Chen Wu, Lei He

arXiv:2610.22897v1cs.AIcs.LG

TL;DR

A failure record may help early in an execution but become redundant later, while still consuming prompt tokens. RECAP measures matched edits against KEEP, proposes them from prefix-only execution relations, and selects policies through independent task-bundle risk tests. Its fixed-block case shows state-dependent value, and its evaluations report backbone-specific reliability-efficiency tradeoffs.

  • Problem

    The value of a failure record depends on what the current execution still needs, but recency and similarity do not show what editing it would do to subsequent success and cost.

    Source: §1 · p. 1

  • Method

    RECAP measures matched KEEP, COMPRESS, and DROP continuations, uses prefix-only execution relations to propose edits, and tests independent task bundles to select policies under two risks.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    At step 15, KEEP succeeds while COMPRESS and DROP fail in all five seeds; at step 27, the same block’s COMPRESS treatment saves 1,908 continuation prompt tokens.

    Source: §1 · p. 1

  • Takeaways & Limitations

    A fixed-block longitudinal case and five-backbone comparison characterize state-dependent intervention value and backbone-specific reliability-efficiency tradeoffs.

    Source: §1 · p. 2, §7 · p. 10

    The study evaluates one edit per matched continuation, and certification assumes iid task bundles under the same fixed sampling and finite-seed measurement protocol.

    Source: §1 · p. 2, §6 · p. 9

Abstract

from arXiv · show

Long-horizon agents learn from failure, but the value of a failure record can change as execution progresses. We study when retaining, compressing, or removing the same history block improves subsequent execution relative to full-history KEEP. We introduce RECAP, a framework that combines matched context interventions, execution-relational prediction, and independent policy certification. Matched environment states and rollout seeds measure success and continuation prompt-cost differences; prefix-only relations between a failure block and the current execution propose promising edits. Independent task bundles then select a policy subject to harmful-intervention and non-positive-benefit risk constraints under a fixed measurement protocol. A five-seed longitudinal case holds both the failure block and its summary fixed: compression fails at an early state and becomes a cost-saving intervention at a later state. Across five backbones, RECAP produces positive paired-success saving estimates with backbone-specific reliability-efficiency tradeoffs. On Hunyuan-A13B, it edits 43.24% of retained checkpoints while matching KEEP's observed 91.89% continuation success and reducing unconditional mean continuation prompt cost by 4,619.8 tokens (2.13%). The framework provides an execution-level reference for deciding when to forget: retain history by its value to what comes next.

1 INTRODUCTION

A failure record can remain useful while its value to the remaining execution changes, so recency or semantic similarity alone cannot determine whether to retain, compress, or remove it. RECAP measures edits against KEEP and selects policies using independent risk tests.

1 INTRODUCTION

  • Failure records can prevent repeated search, expose unmet constraints, and preserve recovery clues, yet may stop affecting execution while continuing to consume prompt tokens.Record value depends on what the current execution still needs; recency and similarity do not directly answer what an edit would do to subsequent success and cost.Source: §1 · p. 1
  • At step 15, KEEP succeeds while COMPRESS and DROP fail in all five seeds; at step 27, COMPRESS saves 1,908 continuation prompt tokens while recovering in two steps.The same failure block and summary are held fixed across checkpoints; recovery means the first valid check of the target slot, not task termination.Source: §1 · p. 1, Figure 1 · p. 2
  • RECAP uses prefix-only execution relations to propose edits, then tests independent task bundles against harmful-intervention and non-positive-benefit constraints, with KEEP as fallback.Matched KEEP/COMPRESS/DROP continuations measure success and cost effects for one identified failure block at a fixed checkpoint.Source: §1 · p. 2
  • The study evaluates one intervention per matched continuation, providing a certificate only for the declared task-bundle protocol.A fixed-block longitudinal case and a five-backbone, seven-selector comparison characterize changing intervention value and reliability-efficiency tradeoffs.Source: §1 · p. 2

2 RELATED WORK

Prior work addresses context representation, memory management, experience reuse, and risk-controlled selection. RECAP focuses on evaluating whether a specified failure-record edit helps a matched continuation.

2 RELATED WORK

  • Prompt-compression and memory systems reduce, represent, or store context, while RECAP evaluates when a specified failure-block edit helps subsequent execution.Examples include selective retention and question-aware compression (Li et al., 2023; Jiang et al., 2023; 2024; Pan et al., 2024), compact representations, and memory systems (Packer et al., 2023; Chhikara et al., 2025).Source: §2 · p. 2
  • Experience-guided agents and learned memory-editing methods use interaction or accumulated experience, whereas RECAP matches the environment and compares retaining, compressing, or removing a selected record.Related approaches include ReAct, Reflexion, Self-Refine, and Voyager (Wang et al., 2023), as well as trajectory condensation and memory operations (Sun et al., 2025; Ye et al., 2025; Yu et al., 2026).Source: §2 · p. 3
  • Risk-control methods calibrate specified losses or prompt choices; RECAP uses Learn-Then-Test with one-sided Clopper–Pearson bounds to select from a development-fixed policy sequence.Its two losses distinguish harmful success changes from non-positive aggregate token benefit (Angelopoulos et al., 2021).Source: §2 · p. 3
  • Unlike off-policy methods that estimate full-policy returns from observational action logs, RECAP directly evaluates all three context treatments at each retained checkpoint.Experiments use configurable planning environments from AgentCE-Bench (Yang et al., 2026).Source: §2 · p. 3

3 SINGLE-CHECKPOINT CONTEXT INTERVENTION

The intervention compares KEEP, COMPRESS, and DROP from the same checkpoint, changing only the specified history block. Its target is continuation success and cost relative to KEEP.

3 SINGLE-CHECKPOINT CONTEXT INTERVENTION

  • KEEP retains the selected block, COMPRESS replaces it with a deterministic summary, and DROP removes its message pairs and matching duplicate failure mentions.All treatments begin from the same checkpoint and leave the true environment state and past execution unchanged.Source: §3 · p. 3
  • Matched seeds compare terminal success and continuation prompt-token cost, with three seeds in the main comparison and five in the illustrative case.The matching protocol replays checkpoints, verifies tool and block identity, and reuses the same frozen checkpoint–action matrix for every selector.Source: §3 · pp. 3–4
  • A finite-seed oracle admits COMPRESS or DROP only when recorded success does not decrease and continuation cost falls, retaining KEEP otherwise.Cost includes the remaining continuation, so shorter prompts at the branch point may be offset by repeated search or recovery.Source: §3 · p. 3
  • The optimal treatment can change with execution state even when the failure block and its compressed replacement remain identical.A rule using only the block text cannot be optimal at two checkpoints when their optimal-action sets are disjoint.Source: §3 · p. 4

4 RECAP: PROPOSAL AND INDEPENDENT CERTIFICATION

RECAP separates execution-relational proposal from independent policy certification. Prefix information proposes edits; task-bundle tests select policies under two risks and a fixed measurement protocol.

4 RECAP: PROPOSAL AND INDEPENDENT CERTIFICATION

  • RECAP separates development fitting and policy fixing, independent calibration-task selection, and held-out evaluation; predictors are backbone-specific while interventions and tests are shared.The candidate family and order are fixed before calibration.Source: §4 · p. 4

4.1 EXECUTION-RELATIONAL PROPOSAL

  • The prefix-only representation uses execution relations—relevance, state change, resolution, dependencies, and subgoals—rather than only failure-block text.It contains 72 state features plus two action indicators; future continuation outcomes are labels, not inputs, and the relation graph is not an identified causal graph.Source: §4.1 · p. 4
  • The proposal selects the edit with highest predicted benefit and applies both harm and benefit gates; if either fails, it returns KEEP without trying the other edit.The harm score estimates recorded success decrease, while the benefit score predicts normalized continuation savings; these scores are not the independent policy test.Source: §4.1 · p. 5

4.2 TASK-BUNDLE RISKS

  • The task-bundle harm loss flags any selected checkpoint with measured success decrease, while the cost loss flags non-positive aggregate normalized continuation savings.The tests are complementary; tasks with no selected edits contribute zero losses but no intervention coverage, and independent tasks are the calibration units.Source: §4.2 · p. 5

4.3 FIXED-SEQUENCE POLICY SELECTION

  • Coverage is the fraction of retained checkpoint decisions differing from KEEP, and the fixed ordered search stops at the first failed candidate.With α = 0.05 and δH = δC = 0.20, selection returns the last accepted candidate with calibration edits, or KEEP if none exists.Source: §4.2 · p. 5, §4.3 · pp. 5–6
  • Under iid calibration and future task bundles sampled and measured by the same fixed protocol, certification bounds both policy-level risks with probability at least 1 − α.The guarantee concerns task-level events under that protocol and does not require predictive scores to be calibrated probabilities.Source: §4.3 · p. 6

5 EXPERIMENTS

The experiments evaluate backbone-specific editing policies against KEEP using matched continuations and task-level risk checks. Across backbones, RECAP shows positive paired-success saving estimates, while a fixed-block trajectory demonstrates that intervention value changes with execution state.

5.1 PROTOCOL, COMPARISONS, AND METRICS

  • The five-backbone evaluation uses separately fitted and calibrated policies on screened task pools; selectors share treatment implementations and a frozen outcome matrix, while metrics include continuation success, task harm, switching, and savings.Unconditional saving includes failures and KEEP decisions; paired-success saving averages over jointly successful pairs.Source: §5.1 · p. 6

5.2 HELD-OUT RELIABILITY–EFFICIENCY TRADEOFFS

  • Across five backbones, RECAP has positive paired-success saving estimates, with observed continuation success within 1.6 percentage points of KEEP; Always-COMPRESS and Always-DROP reduce success on every backbone.Selective policies therefore occupy different reliability–efficiency operating points.Source: Table 1 · p. 7, §5.2 · p. 7, Table 2 · p. 8
  • On Hunyuan, RECAP edits 16 of 37 checkpoints (43.24%), matches KEEP’s 91.89% observed continuation success, and saves 4,619.8 prompt tokens per continuation (2.13%).The unconditional mean prompt cost decreases from 217,219.0 to 212,599.2 tokens; the frozen harm score is constant 1/130, so selection comes from the benefit predictor.Source: §5.2 · p. 7
  • RECAP’s operating points vary by backbone: Qwen3-4B and Qwen3-8B save +10.7 and +32.9 paired tokens, while Qwen3-14B preserves 97.5% success and saves 801 paired tokens.Refitting and recalibrating for Hunyuan raises coverage from 2.70% under a Qwen-derived policy to 43.24%, with unchanged observed success on that matrix.Source: §5.2 · pp. 7–8
  • On 30 Hunyuan calibration tasks, the selected policy’s harm and non-positive-benefit risks are 0.1157 and 0.1722, both below the 0.20 budgets; independent test outcomes then evaluate the frozen rule.All five distinct threshold pairs pass both tests, and the predefined search order selects the final pair, which has the highest coverage but not the largest calibration-sample saving.Source: §5.2 · p. 8

5.3 STATE DEPENDENCE AND REPRESENTATION DIAGNOSTICS

  • At step 27, with the failure block and summary fixed, COMPRESS preserves observed success and reduces mean continuation cost from 23,754 to 21,846 tokens (8.03%); at step 15, both edits fail despite shorter initial prompts.At step 21, DROP saves 154 tokens but requires two additional recovery steps; the measured cost and recovery-first objectives select different actions.Source: Figure 3 · p. 9, §5.3 · p. 9
  • In a 39-task, 286-state diagnostic with 2,574 matched rollouts, execution-relational features improve harm-prediction ROC–AUC to 0.7015 from 0.6147 for structured features; a relational random forest reaches 0.7783 for token-beneficial intervention prediction.This leave-one-task-out effect-prediction diagnostic differs from calibration, which evaluates complete-policy losses.Source: §5.3 · p. 9

6 DISCUSSION

RECAP’s certificate applies to a specified task-bundle protocol and measurement setup, with several boundaries on what its risks and savings represent.

6 DISCUSSION

  • Certification assumes iid task bundles under the same sampling and finite-seed measurement protocol.The certificate applies to that declared protocol.Source: §6 · p. 9
  • The benefit test controls the probability of non-positive aggregate normalized benefit, not loss magnitude or expected raw-token saving.Checkpoint savings are not episode-level savings.Source: §6 · p. 9
  • Reported efficiency is continuation prompt cost; feature extraction, text transformation, and controller runtime overhead are not separately itemized.Summaries use deterministic templates.Source: §6 · p. 9

7 CONCLUSION

The conclusion frames failure-history value in terms of its contribution to what follows in an execution. RECAP operationalizes this idea and reports changing intervention value in a fixed-block case and backbone-specific tradeoffs across five backbones.

7 CONCLUSION

  • Failure history should be assessed by its contribution to subsequent execution, rather than only by age or textual relevance.The paper presents this as an execution-level reference for deciding when to retain history or use a shorter representation.Source: §7 · pp. 9–10
  • RECAP combines matched context interventions, execution-relational proposals, and independent certification of two task-bundle risks.The conclusion describes these as a concrete basis for deciding when retention remains useful and when a shorter representation is sufficient.Source: §7 · p. 10
  • A five-backbone evaluation identifies reliability–efficiency tradeoffs, while Hunyuan-A13B shows unchanged observed continuation success with a 2.13% unconditional prompt-cost reduction.The fixed-block longitudinal case demonstrates changing intervention value.Source: §7 · p. 10

STATEMENT ON THE USE OF AI TOOLS

The authors disclose AI assistance with manuscript and analysis-related tasks while retaining responsibility for the paper.

STATEMENT ON THE USE OF AI TOOLS

  • Generative AI tools assisted with language polishing, manuscript editing, reference organization, LaTeX debugging, and figure and data-analysis code preparation.The disclosure lists assistance across writing, references, figures, and code.Source: STATEMENT ON THE USE OF AI TOOLS · p. 11
  • The authors retain full responsibility for the methods, proofs, experimental results, references, and final manuscript.Responsibility is explicitly assigned to the authors.Source: STATEMENT ON THE USE OF AI TOOLS · p. 11
  • The statement covers both language and technical preparation tasks, but assigns responsibility for the final work to the authors.This summarizes the disclosure’s stated assistance and responsibility boundary.Source: STATEMENT ON THE USE OF AI TOOLS · p. 11

REPRODUCIBILITY STATEMENT

The reproducibility materials specify a frozen, prefix-only checkpoint policy and a fixed independent calibration sequence, alongside Hunyuan’s evaluation protocol and supporting documentation.

REPRODUCIBILITY STATEMENT

  • The reproducibility statement maps the framework’s operational protocol, fixed-block case, task splits, and frozen policy identifiers to the paper’s appendices.Appendix D.2 specifies the feature dictionary, transformations, training targets, model construction, and candidate order.Source: REPRODUCIBILITY STATEMENT · p. 11

A METHOD CONFIGURATION AND OPERATIONAL PROTOCOL

A.1 POLICY EVALUATION AT ONE CHECKPOINT

  • At each checkpoint, the policy computes prefix-only features, scores both edits, applies the selected edit only if both gates pass, and otherwise returns KEEP.It continues the frozen backbone from that checkpoint; each invocation makes one intervention rather than a sequence of checkpoint decisions.Source: §A.1 · p. 14

A.2 INDEPENDENT POLICY CERTIFICATION

  • Independent certification processes candidates in a fixed order and returns the last stored policy that passes both risk budgets, retaining KEEP if none qualifies.The procedure fixes preprocessing, models, thresholds, candidate order, and tie rules before reading calibration outcomes.Source: §A.2 · pp. 14–15

B HUNYUAN EVALUATION PROTOCOL AND DETAILED RESULTS

  • The Hunyuan evaluation uses three matched seeds per retained checkpoint, and its seven selectors reuse the same counterfactual outcomes and KEEP references.Reported results include registered-task and retained-checkpoint counts and the selected calibration bounds.Source: B HUNYUAN EVALUATION PROTOCOL AND… · p. 15

B.1 REGISTERED TASKS VERSUS RETAINED-STATE TASKS

The Hunyuan ledgers distinguish registered-task from retained-state denominators, report test costs and savings, document calibration selection, and compare zero-shot transfer with Hunyuan-specific fitting and calibration.

B.1 REGISTERED TASKS VERSUS RETAINED-STATE TASKS

  • Hunyuan has 19 registered test tasks, but only 15 have retained states; four registered tasks have empty bundles.The registered-task denominator counts zero selected-edit losses for tasks without retained checkpoints, not additional continuation evidence.Source: §B.1 · p. 15, Table B.1 · p. 15
  • For zero observed task harms, the pointwise 95% upper limit is approximately 14.6% over 19 registered tasks and 18.1% over 15 retained tasks.These are deterministic pointwise calculations, not simultaneous statements over methods or backbones.Source: §B.1 · p. 15

B.2 SEVEN-METHOD COST AND ACTION LEDGER

  • RECAP’s paired saving is 5,000.4118 over 102 jointly successful pairs, while unconditional prompt-token saving is 4,619.8288 per continuation.The paired average includes zero differences for KEEP decisions and must not be multiplied by the 16 interventions to estimate total saving.Source: §B.2 · p. 16, Table B.2 · p. 16

B.3 CANDIDATE CALIBRATION LEDGER

  • The selected Hunyuan calibration pair is (λ, τ) = (0.0076923077, 0), and the fixed-order rule returns the final accepted candidate, not the largest-saving candidate.The five distinct candidates pass both 0.20 risk tests; repeated quantile labels do not represent additional distinct policies.Source: §B.3 · pp. 16–17, Table B.3 · p. 16, Figure B.1 · p. 17

B.4 CROSS-BACKBONE TRANSFER DIAGNOSTIC

  • Hunyuan-specific fitting and calibration raise the edited-state share from 2.70% to 43.24% and paired-success saving from 5.9 to 5,000.4 tokens without changing observed success in this matrix.The comparison does not isolate refitting from recalibration.Source: §B.4 · p. 17

C CERTIFICATION DETAILS AND PROOF

This section formalizes fixed-sequence risk certification and documents how execution prefixes, context interventions, checkpoint sampling, and failure-block edits are represented and evaluated. It covers benefit and risk estimation, Hunyuan policy testing, metric distinctions, and a longitudinal case showing that the same failure block can have different measured effects at different checkpoints.

C CERTIFICATION DETAILS AND PROOF

  • With 30 task bundles and γ = 0.025, the upper limits were 0.1157033082 for zero losses and 0.1721694556 for one loss; the fixed-sequence proof bounds inadmissible selection by α.The guarantee assumes independent, identically distributed task bundles and a fixed sample-size protocol; it concerns measured finite-seed outcome differences, not latent task-success probabilities or sequential curation.Source: C CERTIFICATION DETAILS AND PROOF · pp. 17–18

C.1 INTERPRETATION OF THE COVERAGE OBJECTIVE

  • The implemented search uses a development-fixed candidate order and returns its last accepted member with nonzero calibration edits, or KEEP, without requiring nested two-threshold families.The motivating population tradeoff is distinct from this implemented fixed-sequence procedure.Source: §C.1 · p. 18

C.2 A TEXT-ONLY DECISION OBSERVATION

  • If the optimal-action sets for the same block at two checkpoints are disjoint, no deterministic rule based only on that block can be optimal at both checkpoints.This motivates conditioning on execution but does not establish that any particular feature representation is sufficient.Source: §C.2 · p. 18

D COUNTERFACTUAL EVALUATION AND FEATURE DETAILS

  • Prefix replay verifies tool names, arguments, observations, environment-state hashes, and block identity before branching; COMPRESS and DROP change the selected context block, not the environment or historical actions.All selector baselines use the same K/C/D outcome matrix, so KEEP reuses the identical reference outcome; continuations follow a fixed backbone and seed protocol.Source: D COUNTERFACTUAL EVALUATION AND… · p. 18

D.1 CONFIGURATION OF THE FIXED-BLOCK CASE

  • The fixed-block case holds six failed slot checks and their compressed replacement constant across seven checkpoints, with matched environment hashes and successful prefix-output matching.It is a within-task case study, not an estimate of prevalence; the step-15 edits exhausted their 35-step continuation limits without recovery.Source: §D.1 · p. 19
  • At step 27, COMPRESS preserved success in all five seeds and reduced mean continuation prompt cost from 23,754 to 21,846 tokens (8.03%), while DROP cost 3.45 times KEEP.At step 15, both edits failed in all five seeds; at step 21, DROP saved 154 tokens but required two additional recovery steps, so the token-based and recovery-first diagnostics selected different actions.Source: §D.1 · p. 19

D.2.1 THE ORDERED 72-FEATURE REPRESENTATION

  • The ordered representation has 72 prefix-derived relational features plus two action indicators, combining overlap/relevance, temporal/state-change, and dependency-graph features.Resolution, retry, and supersession are operational prefix-derived indicators rather than proof that a failure record is dispensable; the feature dictionary is shown in Table D.1.Source: §D.2.1 · pp. 20, 23, Table D.1 · p. 21

D.2.2 FAILURE BLOCKS, CHECKPOINT SAMPLING, AND SUMMARIES

  • The generic v2 builder retrospectively samples checkpoints from completed successful baselines with at least two detected natural failures, using a default cap of eight and a Hunyuan cap of three.Selected checkpoint features still use only the prefix available at that checkpoint; this is not an online checkpoint scheduler.Source: §D.2.2 · p. 23
  • Each checkpoint’s failure block uses the latest two or three detected failures as adjacent tool-call/output pairs; COMPRESS replaces earlier pairs with deterministic compact JSON, while DROP also scrubs duplicate failure mentions.COMPRESS uses no summary-model call or separate summary-model randomness; DROP removes matching failure fragments from remaining assistant content and reasoning content.Source: §D.2.2 · pp. 23–24

D.2.3 SUPERVISION, NORMALIZATION, AND THE COST-RISK ESTIMAND

  • The forest regresses the ratio of seed-averaged costs, trains on harmful and non-harmful edits including failed continuations, and certifies nonpositive aggregate normalized benefit—not necessarily raw-token saving.Positive normalization preserves an individual checkpoint advantage’s sign but need not preserve the sign of the sum across checkpoints with different KEEP costs.Source: §D.2.3 · p. 24

D.2.4 FINAL PREDICTOR CONSTRUCTION AND THE HUNYUAN INSTANCE

  • Hunyuan’s 64 development edit rows had all-zero harm labels, making harm scores constant; benefit predictions and thresholds drive state-level selection, with risk bounds tested separately on calibration tasks.All Hunyuan risk quantiles coincide, so harm quantiles do not distinguish states.Source: §D.2.4 · p. 25

D.2.5 CANDIDATE GENERATION, ORDERING, AND FROZEN TESTS

  • All 45 Hunyuan candidate tests passed, selecting q01 tau0.00; its 10 calibration edits yielded zero harmful-task flags and one nonpositive-normalized-benefit task.The 30 registered calibration tasks included 15 with empty bundles; the 41 state decisions saved 97,570.67 raw prompt tokens across matched checkpoint continuation means, not one sequential episode.Source: §D.2.5 · p. 26

E METRIC DEFINITIONS AND DIAGNOSTIC PREFERENCE AGREEMENT

  • Unconditional saving averages cost differences across all checkpoint-seed pairs, whereas paired saving conditions on joint successes and is not unconditional expected saving; DPA is distinct from empirical-oracle agreement and coverage.These are prompt-cost measures, not overhead-adjusted wall-clock, monetary, or episode costs; zero DPA does not imply every edit is harmful or non-beneficial.Source: E METRIC DEFINITIONS AND DIAGNOSTIC… · pp. 26–27

E.1 DEVELOPMENT DIAGNOSTIC AND REPRESENTATION RESULTS

The development diagnostic reports treatment variation across tasks and states, while representation diagnostics compare execution-relational features with alternatives. Domain-level results have specific aggregation and backbone-comparison boundaries.

E.1 DEVELOPMENT DIAGNOSTIC AND REPRESENTATION RESULTS

  • Across 39 development tasks, 286 states, and 2,574 matched rollouts, success differs across treatments in 90 states; in the remaining 196, success agrees but efficiency differs.Diagnostic preference ranks KEEP/COMPRESS/DROP in 94/48/144 states and varies within 38 of 39 tasks; these task-level statistics do not hold the failure block fixed.Source: §E.1 · p. 27
  • Execution-relational features reach 0.7015 harm ROC–AUC versus 0.6147 for structured-only features with the same classifier family; a relational forest reaches 0.7783 ROC–AUC for token-beneficial intervention prediction.The semantic-plus-structured MLP reaches 0.4997, but that comparison changes both features and estimator; these are diagnostic results, not evidence of final-predictor sufficiency.Source: §E.1 · p. 27

F AVAILABLE DOMAIN-LEVEL RESULTS

  • Per-domain continuation success is reported as percentages ranked within each backbone and domain; Hunyuan has no recorded per-domain breakdown and appears only in the aggregate comparison.Overall success need not equal an unweighted mean of domain columns because checkpoint counts differ, and backbone results are within-backbone comparisons rather than a controlled scaling-law experiment.Source: Table F.1 · p. 28, F AVAILABLE DOMAIN-LEVEL RESULTS · p. 28

G.1 THE SEPARATE 69-TASK QWEN3-8B CALIBRATION

The standalone Qwen3-8B calibration reports selected-policy outcomes and calibration-size sensitivity, alongside history-length diagnostics and uncertainty caveats. The reported studies remain distinct from the five-backbone comparison and its task-bundle certification scope.

G.1 THE SEPARATE 69-TASK QWEN3-8B CALIBRATION

  • The selected (λ, τ) = (0.36638, 0) policy has five harmful-task and five cost-failure outcomes among 69 tasks; it selects 109 edits and saves 167,799 prompt tokens across matched continuations (363.99 per state).These describe the calibration sample used for selection, not held-out performance or the saving of one sequential episode; the standalone study is separate from the 58-task backbone-adaptation calibration.Source: §G.1 · p. 28, Table G.1 · p. 29

G.2 CALIBRATION-SIZE SENSITIVITY

  • At calibration sizes 10/20/30/40/50/60/69, non-trivial certified-policy fractions are 0/97.2/100/100/100/100/100%, while median switch rates are 0/2.8/7.5/9.4/20.1/21.8/23.6%.The figure draws 500 task subsets below size 69 with predictors and candidate order fixed; this observed-pool diagnostic does not justify collecting tasks until a preferred policy passes.Source: Figure G.1 · p. 29, §G.2 · p. 29

G.3 HISTORICAL STATISTICAL AND HISTORY-LENGTH REPORTING

  • Paired-success token savings across short/medium/long histories are 855/152/1,200 for Qwen3-14B and −53/521/106 for Qwen3-8B; the pattern is not monotone.Bins count prior failures: 2–3, 4–5, and at least 6; bins with fewer than ten eligible events are omitted, and Hunyuan has no recorded breakdown.Source: §G.2 · p. 29, §G.3 · p. 29
  • Historical bootstrap intervals are [−3,336, 2,472] tokens for Qwen3-8B and [280, 1,479] for Qwen3-14B, but current aggregate records do not verify their exact correspondence to current estimates.Resampling should preserve each task’s checkpoint–action–seed bundle; zero-event bootstrap samples may yield [0, 0], which is not zero population risk.Source: §G.3 · pp. 29–30

H.1 EVIDENCE MAP AND VERSION BOUNDARIES

  • The evidence map keeps studies separate rather than pooling their reported quantities, distinguishing the main comparison chain from earlier standalone and diagnostic studies.The main chain runs from screened task split and frozen matched outcomes through development-fitted scores and calibration to held-out test selection.Source: §H.1 · p. 30, Table H.1 · p. 30

H.2 MAIN-COMPARISON SPLITS

  • In the five-backbone comparison, Hunyuan development/calibration/test contain 14/15/15 tasks with retained states.The corresponding Hunyuan counts are 32/41/37 states and 288/369/333 counterfactual rollouts; the seven selectors reuse these continuations.Source: Table H.2 · p. 30, §H.2 · p. 30

I EXTENDED RELATED WORK

  • RECAP complements context-compression and memory systems by evaluating selected failure-block edits from matched checkpoint continuations and separately certifying task-bundle risks.The frozen single-checkpoint action matrix does not compare end-to-end memory or skill-management policies, which require separate rollouts; the certificate is not a guarantee for repeated edits in an evolving episode.Source: I EXTENDED RELATED WORK · pp. 31–32
Loading 2610.22897v1…