PaperlayerView exact revision on arXiv

Source-linked AI summary

FSPO: Policy-Consistent Risk and Pareto-Feasible Control for Budgeted LLM RL Post-Training

Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Daren Zha, Jun Xiao

arXiv:2610.02828v1cs.AIcs.CLcs.LG

TL;DR

Adaptive post-training changes multiple actuators, but future-risk estimates, calibration after action selection, and multi-resource feasibility may not match the deployed controller’s decisions. FSPO combines policy-consistent risk-to-go, decision-conditioned trajectory calibration, and Pareto resource continuation for budgeted LLM RL post-training. Under a matched GRPO resource envelope, it outperforms PB2 on held-out and OOD accuracy.

  • Problem

    Adaptive post-training faces mismatches between behavior-policy risk and deployment risk, logged-pair and selected-action calibration, and independent resource minima and feasible multi-resource continuations.

    Source: §1 · pp. 1–2

  • Method

    FSPO aligns utility and future-risk targets with a frozen controller, calibrates selected decisions on cross-fitted trajectories, and admits actions only when a Pareto resource continuation remains feasible.

    Source: §1 · p. 2, §3.4 · p. 4, §3.6 · p. 5

  • Results

    66.11% held-out and 59.43% OOD accuracy for FSPO, versus 64.47% held-out and 57.03% OOD accuracy for PB2, the strongest evaluated baseline.

    Source: §4.2 · p. 6

  • Takeaways & Limitations

    In the reported factorial ablation, enabling all three components reduces trajectory failure from 0.181 to 0.083.

    Source: §6 · p. 8

    PRCC certifies resource feasibility under deterministic reservations and persistent catalog availability; runtime failures, stochastic costs, or a changing catalog require a refreshed frontier.

    Source: §5 · p. 8

Abstract

from arXiv · show

Adaptive LLM reinforcement-learning post-training changes multiple training actuators online, including rollout temperature, group size, clipping, KL regularization, verifier allocation, and update budget. Three coupled issues remain unresolved. A future-risk model trained from behavior trajectories need not estimate the risk induced by the controller that will be deployed; a score calibrated on logged state-action pairs can become miscalibrated after selective action choice; and independent per-resource minimum costs do not in general certify a feasible multi-resource continuation. We introduce FSPO, a feedback-state controller for budgeted LLM RL post-training that addresses these issues jointly. FSPO learns a policy-consistent risk-to-go model whose Bellman target follows the same frozen controller used for future decisions, together with a long-horizon utility model. Decision-conditioned trajectory calibration (DCTC) calibrates risk on cross-fitted trajectories generated by actions selected by provisional controllers. A Pareto resource continuation certificate (PRCC) admits an action only when a non-dominated cumulative reservation remains feasible over the residual horizon. Under a matched GRPO resource envelope, FSPO reaches 66.11% held-out and 59.43% OOD accuracy, compared with 64.47% and 57.03% for PB2, the strongest evaluated adaptive baseline. Three paired training seeds give gains of +2.42 and +3.19 percentage points over the contextual bandit on held-out and OOD evaluation. Under high behavior-deployment mismatch, policy-consistent risk lowers selected-decision ECE from 0.108 to 0.053; DCTC lowers it from 0.039 to 0.022 at matched acceptance; PRCC removes false-feasible admissions on an 18-action catalog ($0.197\rightarrow0.000$); and enabling all three components reduces trajectory failure from 0.181 to 0.083 in a factorial ablation.

1 INTRODUCTION

Adaptive post-training control must address future-risk estimates, selected-action calibration, and vector-valued resource feasibility together. FSPO aligns these decisions with a frozen deployment controller and evaluates its quality under a matched GRPO resource envelope.

1 INTRODUCTION

  • Behavior-policy failure labels need not estimate risk under a different deployment controller, logged-pair calibration can shift after action selection, and componentwise cost minima may not yield a feasible continuation.These are policy-induced risk shift, selection-induced calibration shift, and a multi-resource feasibility problem.Source: §1 · pp. 1–2
  • FSPO treats adaptive post-training as constrained sequential control, aligning utility and future risk with a frozen controller, calibration with selected decisions, and admission with a Pareto continuation certificate.Its ledger, fail-closed predicate, and post-freeze oracle make transitions auditable without putting future outcomes in the online state.Source: §1 · p. 2
  • FSPO reaches 66.11% held-out and 59.43% OOD accuracy in the matched-GRPO trainer comparison over three independent training seeds.The controlled mechanism benchmark with deterministic replay separately isolates the decision contract.Source: §1 · p. 2
  • The paper contributes controller-consistent risk-to-go, decision-conditioned trajectory calibration, Pareto resource continuation certificates, and matched-budget GRPO evaluation.The replay ledger is reproducibility infrastructure rather than an algorithmic contribution.Source: §1 · p. 2

2 RELATED WORK

Prior work adapts training strategies and allocates compute, while constrained-control and calibration research address related pieces of the problem. FSPO brings these concerns together for budgeted LLM post-training and specifies its calibration splits and evaluation measures.

2 RELATED WORK

  • Population-based, Bayesian, meta-gradient, trust-region, adaptive-KL, bandit, and runtime-system approaches adapt training or allocate compute, but do not by themselves specify FSPO’s joint control concerns.The paper situates its approach alongside work on GRPO, large-scale post-training recipes, and resource orchestration.Source: §2 · p. 2
  • FSPO combines policy-dependent future risk, selected-action calibration, and realizable multi-resource continuation, concerns treated separately across constrained RL, calibration, and multi-objective decision-making.The related work also connects state-conditioned action representations, trust-region quantities, and offline-RL evaluation contracts to the problem.Source: §2 · pp. 2–3

2.1 RISK CALIBRATION AND SELECTIVE CONTROL

  • Calibration uses disjoint complete-trajectory roles and reports both all-action and selected-action errors alongside Brier score, coverage, failure, abstention, and risk–utility frontiers.The protocol keeps future-failure labels from one trajectory within a single split role.Source: §2.1 · p. 3

3 METHOD

FSPO defines an auditable feedback-state control protocol, distinct evaluation estimands, and constrained decisions over a bounded action catalog. Its utility, policy-consistent risk, selected-decision calibration, and Pareto resource certificate jointly govern action selection and continuation feasibility.

3.1 PROBLEM AND PROTOCOL

  • The protocol separates online telemetry and ledger state from post-trace oracle labels and scores, and records run configuration and deterministic replay digests for reproducibility.The online policy selects bounded actions over eight intervals; the separate scoring pass joins clean oracle scores only after decisions are frozen.Source: §3.1 · p. 3, §3.3 · p. 4, Table 1 · p. 4

3.2 EVIDENCE-BOUNDED ESTIMANDS

  • The estimands distinguish declared ledger efficiency from the frozen-trace action gap, with regime and cost-contract contrasts evaluated using paired complete-case bootstrap intervals.The environment fixture, cost contract, and policy define the evaluation setting.Source: §3.2 · p. 3

3.3 CONTROLLER OBJECTIVE AND CONSTRAINTS

  • The controller selects among three bounded actions using telemetry, while a stability predicate determines whether an interval is accepted and committed.The action catalog constrains temperature, group size, clipping, KL coefficient, verifier calls, and escalation; violations contribute zero effective progress.Source: §3.3 · pp. 3–4

3.4 POLICY-CONSISTENT RISK-TO-GO CONTROL

  • FSPO fits long-horizon utility and future stability risk under a frozen controller, with risk recursion following that controller’s future actions rather than a behavior-policy suffix.Provisional controllers select next actions for both targets; utility is pessimistic and risk conservative, with a calibration-frozen map applied to risk for admission.Source: §3.4 · pp. 4–5

3.5 DECISION-CONDITIONED TRAJECTORY CALIBRATION

  • DCTC fits calibration on cross-fitted trajectories replayed under held-out provisional controllers, retaining selected decisions and complete-trajectory labels to align calibration with deployment choices.The risk threshold is selected on the same decision-conditioned calibration role under a predeclared coverage–risk criterion.Source: §3.5 · p. 5

3.6 PARETO RESOURCE CONTINUATION CERTIFICATE

  • PRCC admits an action only when a non-dominated cumulative reservation leaves a feasible residual resource plan; with deterministic reservations and a persistent catalog, this certifies a feasible continuation sequence.The certificate concerns resource feasibility, not future-risk gating; if no action is feasible, FSPO abstains and executes no rollout or optimizer update.Source: §3.6 · p. 5

4 EXPERIMENTS

The experiments evaluate FSPO on a controlled mechanism benchmark and a matched-budget GRPO study. FSPO leads the evaluated controllers on GRPO accuracy, while component tests examine calibration, resource feasibility, and feedback value.

4.1 EXPERIMENTAL SETUP

  • The GRPO study uses Qwen2.5-3B-Instruct, three training seeds, disjoint calibration and evaluation stream seeds, and a matched resource envelope.Each policy receives 12.583M training tokens, 1,536 optimizer steps, and 9.60 GPU-hours per seed.Source: §4.1 · pp. 5–6, §4 · p. 5

4.2 MAIN RESULTS

  • FSPO reaches 66.11% held-out and 59.43% OOD accuracy, versus PB2’s 64.47% and 57.03%, at 9.29 training GPU-hours per seed.Its interval failure is 0.014, 0.011 below the same-architecture Safety-Critic Gate.Source: §4.2 · p. 6, Table 2 · p. 7
  • At high behavior–deployment mismatch, policy-consistent recursion has selected-decision ECE 0.053 versus 0.108 for behavior-policy Monte-Carlo risk; DCTC lowers selected-action ECE from 0.039 to 0.022.DCTC’s reduction is reported at matched acceptance.Source: §4.2 · pp. 6–7, Figure 2 · p. 7
  • PRCC reduces false-feasible admissions from 0.197 to 0.000 on the 18-action catalog; the full three-component combination lowers trajectory failure from 0.181 to 0.083.The full combination adds 3.17 held-out accuracy points, exceeding the 1.98-point sum of single-component gains.Source: Figure 2 · p. 7, §4.2 · p. 7
  • Across five fixture/transition streams, the fixed controller averages 2.18858 progress per ledger unit, while the rule averages 1.35100.On the shared 24-case, eight-interval contract, the rule leads at 2.8283, but its cross-stream sample SD is 0.84832.Source: §4.2 · p. 7, Table 3 · p. 8

4.3 ABLATIONS AND MECHANISM ANALYSIS

  • The calibration-only threshold sweep favors the matched controller over default and frozen-threshold rules on held-out and OOD mean case progress, while reward choice changes progress per ledger unit.The frozen η-aligned bandit reward lowers that metric by 0.9901 on held-out and 1.0058 on OOD versus the step-ratio reward.Source: §4.3 · p. 8
  • Freezing or delaying telemetry lowers GRPO accuracy relative to FSPO, and feedback controls raise violations on the mechanism benchmark.Freezing telemetry lowers FSPO to 62.77%/54.98% and raises interval failure from 0.014 to 0.043.Source: §4.3 · p. 8

5 LIMITATIONS

The reported feasibility certificate depends on deterministic reservations and persistent catalog availability, and transfer is strongest with target-block refitting and recalibration.

5 LIMITATIONS

  • PRCC certifies resource feasibility under deterministic reservations and persistent catalog availability.Runtime failures, stochastic costs, or a changing catalog during an interval require a refreshed frontier.Source: §5 · p. 8
  • Transfer across tasks and model scales is strongest when utility and risk models are refit and recalibrated in the target block.Fully frozen transfer raises 7B-block trajectory failure from 0.126 to 0.204.Source: §5 · p. 8
  • Decision latency grows from 0.73 to 2.06 ms as the action catalog grows from 3 to 18 actions.Source: §5 · p. 8

6 CONCLUSION

FSPO combines policy-consistent future risk, selected-action calibration, and a Pareto certificate for multi-resource continuation. Under a matched GRPO contract, it outperforms PB2 and the contextual bandit on held-out and OOD accuracy.

6 CONCLUSION

  • 66.11%/59.43% held-out/OOD accuracy at 9.29 training GPU-hours per seed, versus 64.47%/57.03% for PB2 and 63.69%/56.24% for the contextual bandit.Source: §6 · p. 8
  • Enabling all three components lowers trajectory failure from 0.181 to 0.083 in stress tests.Source: §6 · p. 8
  • FSPO records the lowest interval failure (0.014) under the matched GRPO contract.Source: §6 · p. 8

AI-USE STATEMENT

The authors used generative AI for language polishing and reference summaries, while the methodology, experiments, analysis, and interpretation were conducted and verified by the authors.

AI-USE STATEMENT

  • Generative AI tools were used for language polishing and summarizing references.Source: AI-USE STATEMENT · p. 9
  • The authors did not use generative AI to generate results or synthetic datasets, formulate mathematical claims, provide proofs, or decide research conclusions.Source: AI-USE STATEMENT · p. 9
  • The authors conducted and verified the methodology, experimental setup, analysis, and interpretation, and take responsibility for the final content and artifacts.Source: AI-USE STATEMENT · p. 9

ETHICS STATEMENT

The work evaluates resource-aware control of language-model post-training with auditable benchmark workloads and controlled mechanism evaluations, without human participants or personal data.

ETHICS STATEMENT

  • The study uses auditable benchmark workloads and controlled mechanism evaluations for resource-aware language-model post-training control.Source: ETHICS STATEMENT · p. 9
  • The work uses no human participants, personal data, or sensitive user records.Source: ETHICS STATEMENT · p. 9
  • The evaluation records failure and abstention outcomes, and authors advise reviewing deployment-specific safety requirements before live use.Source: ETHICS STATEMENT · p. 9

REPRODUCIBILITY STATEMENT

The paper specifies the controller and evaluation contracts and provides supplementary artifacts intended to support independent regeneration and auditing.

REPRODUCIBILITY STATEMENT

  • The paper specifies the state, composite action catalog, risk target, calibration, Pareto certificate, split roles, aggregation units, and matched resource contract.Source: REPRODUCIBILITY STATEMENT · p. 9
  • Supplementary materials provide code, configuration files, result schemas, replay checks, and figure-generation scripts.Source: REPRODUCIBILITY STATEMENT · p. 9
  • Tables bind comparisons to declared seeds, evaluators, cost units, and outcome-row contracts for independent regeneration and auditing.Source: REPRODUCIBILITY STATEMENT · p. 9

A TASK AND METHOD DETAILS

The appendix specifies FSPO’s state, action, fitting, replay, and split contracts, then reports an observational telemetry replay separately from executed mechanism outcomes.

A.1 STATE AND FEATURE REPRESENTATION

  • The online state stores telemetry and ledger information while reserving oracle progress for the post-freeze record, preserving the policy’s online information boundary.The serialized record includes recent progress, reward dispersion, verifier measures, control settings, KL telemetry, budget, and interval index.Source: §A.1 · p. 12, §B.2 · p. 16

A.2 COMPOSITE ACTIONS AND TRANSITION DYNAMICS

  • Eight-interval runs use composite catalog actions and ledger costs; stability violations trigger rollback with zero effective progress, while rejected or over-budget actions fail closed.The declared transition family specifies progress, stability, and state updates; Table 4 lists composite actions and declared ledger costs.Source: §A.2 · pp. 12–13, Table 4 · p. 12

A.3 MECHANISM BASELINE POLICIES

  • The mechanism comparison uses four fully specified policies: only the contextual bandit learns online, and all policies share the same fixture, horizon, and ledger budget.The baselines include uniform, rule-based, feedback-state, and contextual-bandit policies.Source: §A.3 · pp. 13–14

A.4 RISK-CALIBRATED LEARNED CONTROL

  • FSPO fits utility and policy-consistent risk heads against the same provisional controller, calibrates selected decisions on cross-fitted trajectories, and filters actions by residual-budget feasibility.Normalization, models, calibration map, threshold, reward reference, and cost model are frozen before held-out or OOD evaluation.Source: §A.4 · pp. 14–15

A.5 REPLAY AND INFORMATION FLOW

  • Deterministic replay checks event ordering, state transitions, action logs, outcome joins, and trainer records; artifacts retain audit data while excluding weights and large caches from the public bundle.The clean oracle is joined only after the trace digest is frozen.Source: §A.5 · pp. 15–16, §A.4 · p. 15

B.1 SEED ROLES AND SPLIT CONSTRUCTION

  • Calibration, held-out, and OOD streams have separate roles; paired evaluations retain complete cases, and stream efficiency is total progress divided by total ledger cost.OOD streams change regime composition while preserving the action and ledger schema.Source: B DATA AND EVALUATION PROTOCOLS · p. 16, §B.1 · p. 16

B.2 POLICY-VISIBLE STATE DICTIONARY / B.3 ACTION, TRANSITION, AND OUTCOME SCHEMA

  • The state dictionary and event schema enforce which fields are available before action selection and keep action propensities distinct from subsequent outcomes.Target scores, regime identifiers, and clean action outcomes cannot enter the policy input before the event is committed.Source: §B.2 · p. 16, §B.3 · p. 17, Table 7 · p. 17

B.4 OBSERVATIONAL SHADOW PROTOCOL

  • 0.032566 calibration and 0.032135 evaluation signed mean-loss reductions are logged-outcome associations, while shadow recommendations are not executed interventions.The shadow replay uses 100 calibration intervals and 50 evaluation intervals from a fixed external telemetry stream.Source: §B.4 · pp. 17–18, Table 10 · p. 18

B.5 TRAINER WORKLOAD AND EVALUATION PROTOCOL

The trainer protocol fixes the workload, evaluation roles, and resource envelope for GRPO post-training comparisons, with FSPO decisions applied before rollout generation.

B.5 TRAINER WORKLOAD AND EVALUATION PROTOCOL

  • The trainer comparison binds its workload, evaluator, split protocol, methods, and reporting surfaces to a shared event schema and manifest.The held-out split tests structural repair within BFCL; OOD changes the evaluation family while retaining the repair interface.Source: §B.5 · p. 18, Table 11 · p. 18

B.6 GRPO POST-TRAINING PROTOCOL

  • Calibration selects thresholds and controller parameters, while held-out and OOD runs use disjoint seeds under a matched resource envelope.The same source model, prompt set, evaluator versions, maximum tokens, optimizer steps, and GPU-hour envelope apply across policies.Source: §B.6 · p. 18, §B.5 · p. 18, Table 12 · p. 19
  • Actions are applied before rollout generation: temperature, group size, clipping, and KL control generation, group advantages, ratio bounds, and the trust-region penalty, respectively.Verifier allocation supplies verifier views for group reward, while escalation requests additional micro-step levels.Source: §B.6 · p. 19, Table 13 · p. 19, Table 14 · p. 20
  • Trainer progress is normalized monitor-reward change, whereas held-out accuracy is final task quality; stability flags cover KL-cap, numerical, memory, and verifier-service failures.Monitor evaluations and verifier retries consume the shared resource envelope.Source: §B.6 · pp. 19–20

C IMPLEMENTATION AND BASELINES

The implementation fixes an auditable decision and replay contract, then evaluates adaptive control across regime, feedback, risk-calibration, and resource-feasibility analyses. The analyses examine how controller choice, feedback information, and multi-resource constraints affect measured outcomes.

C.1 FULL DECISION ALGORITHM

  • The implementation separates an oracle-independent decision trace from offline diagnostics, joining oracle outcomes only after the trace is frozen.Deterministic replay checks the manifest, transitions, costs, coverage, and final digest.Source: §C.1 · p. 20, §C.5 · p. 22

C.2 BASELINE AND MATCHED-CONTROL CONTRACTS

  • The baseline contracts hold the action catalog, horizon, initial state, budget, and transition family fixed while varying action selection and online updates.Trainer comparisons additionally match the model, optimizer, training data, evaluation cadence, and resource envelope.Source: §C.2 · p. 21, §C.3 · p. 21, Table 15 · p. 21

D.2 REGIME AND THRESHOLD ABLATIONS

  • The frozen threshold remains above the controller on the OOD partition ratio, while the eta-aligned bandit loses more held-out efficiency.The held-out audit also reports a step-ratio advantage for the bandit under its native reward.Source: §D.2 · p. 23, Table 19 · p. 23

D.5 SEED UNCERTAINTY AND REPLAY CONTRASTS

  • The mechanism audit reports stream-level dispersion and paired case-wise contrasts, while trainer measurements keep ledger, hardware, and task-quality units distinct.The held-out/OOD audit freezes its threshold using calibration-only seeds and evaluates on disjoint held-out and OOD seeds.Source: §C.4 · p. 22, §D.5 · p. 24, §D.6 · p. 25

D.8 FEEDBACK INFORMATION AND ADAPTATION CONTROLS

  • Delayed telemetry and fixed feature channels reduce progress and raise violations, while within-regime shuffling produces the largest paired loss.The controls preserve the action catalog and budget while changing the information channel.Source: §D.8 · p. 26

D.9 TRAJECTORY RISK CALIBRATION AND THRESHOLD SWEEP

  • The risk-calibration audit evaluates raw and adjusted risk scores alongside trajectory-blocked calibration, using independent trajectory roles for fitting, calibration, held-out, and OOD data.The threshold sweep is descriptive; its threshold is selected before final evaluation.Source: §D.9 · pp. 26–27

D.10 POLICY-CONSISTENCY STRESS TEST

  • Under high behavior-deployment mismatch, policy-consistent risk has selected ECE 0.053 versus 0.108 and trajectory failure 0.128 versus 0.236 for behavior-policy error.The policy-consistent estimator abstains slightly more often.Source: §D.10 · p. 27

D.12 PARETO CONTINUATION FEASIBILITY STRESS TEST

  • The audit compares one-step admission, independent componentwise-minimum reservation, and PRCC on trade-off catalogs where different actions minimize tokens, verifier calls, and device-time.False-feasible admission means a decision passes admission but has no resource-feasible continuation.Source: §D.12 · p. 28

D.13 THREE-COMPONENT FACTORIAL AND INCREMENTAL ABLATION

The factorial ablation tests policy consistency, decision-conditioned calibration, and Pareto continuation under a matched GRPO contract. The full combination improves held-out and OOD accuracy over the no-component variant and lowers trajectory failure.

D.13 THREE-COMPONENT FACTORIAL AND INCREMENTAL ABLATION

  • The full three-component combination adds 3.17 held-out and 4.56 OOD points over the no-component variant, while lowering trajectory failure from 0.181 to 0.083.Its joint gain exceeds the 1.98-point sum of single-component gains; development cost increases by 0.78 GPU-hours per seed.Source: §D.13 · pp. 28–29, Table 37 · p. 30

D.16 OFFLINE SUPPORT AND EXTRAPOLATION DIAGNOSTICS

  • The offline support audit requires nonzero calibration support for every catalog action and records unseen-action rates, feature-envelope distance, and utility and risk error.The support audit reflects behavior-policy composition rather than assuming counterfactual coverage.Source: §D.16 · pp. 29–30, Table 40 · p. 31

D.17 MATCHED CALIBRATION AND DEVELOPMENT COST

D.18 CROSS-MODEL AND CROSS-TASK TRANSFER

  • Cross-model and cross-task transfer repeats controller fitting and trajectory calibration within each target block rather than copying results across blocks.The study distinguishes refitting, recalibration-only, and fully frozen transfer modes.Source: §D.18 · p. 30, Table 43 · p. 32, Table 42 · p. 32

D.20 MATCHED-BUDGET AND SCALE COMPARISONS

  • At 96 cases, the controller reaches 13.3928 progress units while the rule has the highest ratio, 2.6914; at 192 cases and 16 intervals, the controller reaches 54.1268 while the rule retains the highest ratio, 2.6128.The expanded settings preserve regime proportions, catalog, and calibration contract.Source: §D.20 · p. 32, Table 50 · p. 35

D.22 MATCHED GRPO TASK RESULTS

  • The GRPO study defines held-out accuracy at matched GPU-hours as its primary endpoint and reports OOD accuracy, failure, rollout cost, and action/outcome coverage alongside it.Policy comparisons pair the same training seed, data order, evaluator digest, and resource cap.Source: §D.22 · p. 33, §D.22.1 · p. 34, Table 49 · p. 34

D.23 ACTION-SPACE AND ENVIRONMENT-FAMILY SCALING

  • Expanding the catalog to 9 and 18 actions raises held-out accuracy to 66.83% and 67.02%, at 9.37 and 9.49 GPU-hours per seed.Under shifted dispersion and reliability families, the controller retains a positive paired ledger-ratio advantage over best-static.Source: §D.23 · p. 35, Table 56 · p. 37

D.24 LEARNED-CONTROLLER COMPONENT ABLATIONS

  • Removing the risk filter raises interval failure from 0.014 to 0.052, while the myopic utility target lowers held-out accuracy by 1.54 points at similar cost.The ablation preserves runtime resource enforcement while changing the specified decision rule.Source: §D.24 · pp. 35–36, Table 58 · p. 36

E.1 REPRODUCIBILITY CHECKLIST

The reproducibility checklist binds reported results to declared splits, units, source artifacts, and deterministic checks. Trainer receipts and paired records connect task, resource, and action outcomes to the same run and policy trace.

E.1 REPRODUCIBILITY CHECKLIST

  • Each result is identified by a source-relative run identifier, and hashes bind the raw report, metrics, manifest, and corresponding table inputs.The release manifest records source-relative artifact handles.Source: §E.1 · p. 37, §E.2 · p. 38, Table 59 · p. 38
  • The mechanism benchmark uses one 24-case fixture, while calibration, held-out, and OOD roles use distinct assigned seed sets documented with the fixture digest.The external replay supplies its observational calibration/evaluation stream in its own artifact.Source: §E.1 · p. 37
  • Cost records report mechanism calls, tokens, ledger units, retries, and source timings, while trainer records add optimizer steps, GPU-hours, wall-clock time, memory, and task targets.External GPU-second accounting is joined by split and seed.Source: §E.1 · p. 37

E.4 REPRODUCIBILITY AND RELEASE PLAN

  • The public release includes manuscript and validation materials while omitting contributor and host names, secrets, checkpoints, model weights, and large caches.Versioned manifests bind tables to source-relative inputs.Source: §E.1 · p. 37, §E.2 · p. 38

E.5 CROSS-TABLE CONSISTENCY CHECKS

  • Cross-table checks verify that headline values, seed summaries, partition rows, reward deltas, action propensities, transitions, and paired contrasts agree with their source records.Replay reports the first differing event and field before regenerating a changed summary.Source: §E.5 · p. 39

E.6 TRAINER EXECUTION AND RECEIPT SEQUENCE

  • The trainer protocol uses ordered stages, each producing a receipt before the next stage begins.The primary criterion is paired held-out accuracy improvement at matched GPU-hours with the same direction on OOD seeds.Source: §E.6 · p. 39, Table 61 · p. 39

E.7 RECEIPT INTERPRETATION AND DECISION RECORD

  • Task-quality, resource, and action-mix contrasts join their respective evaluator, scheduler, and event records, with intervals using the same resampling unit as point estimates.The joined decision record distinguishes task-quality gains from ledger-ratio mechanism gains.Source: §E.7 · p. 40
Loading 2610.02828v1…