PaperlayerView exact revision on arXiv

Source-linked AI summary

The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs

Xin Li, Hao Jiang, Annan Wang, Yichi Zhang, Chau Yuen

arXiv:2605.08737v1cs.LGcs.CL

TL;DR

Structured-output OPD can improve domain performance until reward extrapolation crosses a clip-safety threshold, after which the output contract collapses. The paper derives a base-relative threshold from teacher confidence, warm-start mass, and clip strength, extends it to listwise JSON scaffolds, and tests it on Amazon Fashion. Three pre-registered tests match their prediction windows, and operating just below the threshold brings a 1.7B student to parity with an 8B-SFT baseline, primarily through format adherence.

  • Problem

    Reward-extrapolated OPD can lift students past teachers, but its threshold-dependent effect on structured-output contract validity lacks a closed-form boundary.

  • Method

    The paper derives λ⋆(p, b, c) in a Bernoulli reduction and calibrates a sequence-level rule for near-deterministic listwise JSON scaffolds.

  • Results

    Three pre-registered Fashion tests fall within locked prediction windows, while operating below λ⋆ raises USEFUL from 0.23 to 0.86 and matches an 8B-SFT baseline.

  • Takeaways & Limitations

    Parse validity changes sharply at the predicted boundary while NDCG@1 on parsed outputs remains flat, making the cliff primarily a format-adherence phenomenon.

  • Takeaways & Limitations

    The rule is scoped by base relativity, near-deterministic measurable scaffolds, finite-budget reachability, and Gemini exposure for the parity claim.

Abstract

from arXiv · show

On-policy distillation (OPD) is widely used for LLM post-training. When pushed with a reward-extrapolation coefficient lambda > 1, the student can lift past the teacher in domain, but past a threshold lambda* the same step violates the output contract on structured-output tasks. In a single-position Bernoulli reduction, we derive a closed-form base-relative clip-safety threshold lambda*(p,b,c) determined by three measurable quantities: the teacher modal probability, the warm-start mass, and the importance-sampling clip strength. Above lambda*, the extrapolated fixed point exits the clip-safe region, changing training from format-preserving to format-collapsing. We extend the rule to calibrated K-ary listwise JSON tasks where a single binding equivalence class dominates the output contract and SFT retains parse headroom. On Amazon Fashion, three pre-registered tests--a fine-grid cliff interval, a budget-extension test, and a small-clip cross-prediction--fall within their locked prediction windows, with the small-clip value matching the closed-form prediction below grid resolution. Operating just below lambda*, ListOPD brings a 1.7B Qwen3 student to in-domain parity with an 8B-SFT baseline at one-fifth the parameters. The gain is driven primarily by format adherence: NDCG@1 on parsed outputs remains flat across lambda, while parse validity sharply changes at the predicted boundary. The cliff diagnostic is rubric-independent, whereas the parity claim uses a Gemini-graded rubric and inherits that evaluator's exposure.

1 Introduction

The paper identifies a sharp extrapolation threshold beyond which OPD collapses structured-output contracts, then derives and tests a measurable boundary for operating safely. On Amazon Fashion, staying below that boundary improves deployment-useful performance and reaches parity with a much larger SFT baseline.

  • Motivation: OPD reward extrapolation can lift students past teachers, but beyond λ⋆ it changes structured-output training from format-preserving to contract-collapsing.The threshold is derived and calibrated on Amazon product-review listwise ranking.
  • Motivation: A Qwen3-1.7B-SFT student improves contract satisfaction from 33.5% to 94.8% with ListOPD, while rank quality on parsed outputs remains unchanged.Fashion is used as a controlled scaffold for contract-adherence mechanics rather than a semantic-ranking comparison.
  • Results: Operating below λ⋆ raises USEFUL from 0.23 to 0.86 and matches a pre-registered 8B-SFT baseline within combined seed noise.The training-side residual against constrained-SFT plus permutation repair is +0.051 USEFUL.
  • Method: The clip-safety boundary is determined by teacher modal probability, warm-start mass, and IS clip strength, turning λ tuning into a falsifiable prediction problem.The paper links the boundary to the extrapolated fixed point exiting the clip-safe region.
  • Evidence: Three pre-registered Fashion tests match their locked prediction windows, including a fine-grid cliff interval and a small-clip cross-prediction below grid resolution.The tests include a budget extension and support the predicted boundary scale.

2 Related Work

The paper positions its contribution at the intersection of on-policy distillation, structured-output format adherence, and listwise ranking. It distinguishes the derived cliff boundary from mitigation methods and constrained decoding.

  • Distillation and on-policy RL: The work extends reward-extrapolated OPD from reasoning to listwise structured-output ranking and derives the base-relative threshold λ⋆(p, b, c).ASPO addresses the related IS asymmetry with a ratio-flip training fix, whereas this paper derives the boundary where the fixed point becomes unsafe.
  • Distillation and on-policy RL: The cliff is presented as a property of IS-clipped reverse-KL extrapolation rather than of one particular method.ASPO exhibits its own cliff one grid step earlier than vanilla OPD, supporting a mechanism-level interpretation.
  • Format adherence and listwise ranking: Unlike constrained decoding, the paper targets format adherence through training while deriving when that adherence collapses.Strict-K decoders can convert the capability gap into duplicate-id pathology without improving task-level validity.
  • Format adherence and listwise ranking: The paper connects its result to evidence that task-solving and formatting can decouple, while adding a closed-form λ boundary absent from that prior observation.The related work frames the cliff as an on-policy analogue of format-induced training brittleness.

3 Method and Experimental Setup

ListOPD applies base-relative, reward-extrapolated reverse-KL distillation to autoregressive K-item JSON rollouts, training structural and score tokens together. The experiments use SFT-warmstarted Qwen3 models and deployment-oriented parse and ranking metrics.

  • Listwise JSON rollout: A K-item listwise rollout generates JSON objects containing review identifiers and scores autoregressively from the full product prompt.The displayed target contains K objects such as {review_id, score} pairs.
  • Listwise JSON rollout: Structural delimiters and score tokens are interleaved, so token-level reverse-KL gradients distribute credit across both output scaffolding and scores.The same token autoregression mechanically yields the listwise factorization used for ordered scores.
  • On-policy distillation: ListOPD uses a base-relative sharpened teacher target proportional to πB(πT /πB)^λ, with λ=1 recovering vanilla reverse-KL distillation.Training uses GRPO token-level IS correction with clip c=5.0 and no additional KL penalty.
  • Models and data: Qwen3 models at 0.6B, 1.7B, 4B, and 8B are SFT-warmstarted for five epochs on Amazon Fashion PL-K8 data before OPD or baseline comparison.Teacher candidates are 4B and 8B SFT checkpoints.
  • Evaluation: Evaluation enforces exactly K unique review identifiers and numeric scores, reporting parse_rate, ranking metrics on parsable outputs, and USEFUL=parse × NDCG@1.USEFUL is the sole operating-point selection metric; other metrics are diagnostic.

4 Single-Position Threshold and Sequence Calibration

The paper derives a closed-form single-position clip-safety threshold and lifts it to calibrated sequence-level JSON scaffolds under explicit assumptions. Fashion measurements place the observed cliff near the predicted scale, while finite-budget drift and task structure bound the rule’s scope.

  • Single-position threshold: The threshold λ⋆(p, b, c) uses teacher modal probability p, warm-start modal probability b, and IS clip strength c.The notation distinguishes sequence-level ptyp and psafe from their binding-position counterpart beff.
  • Single-position threshold: Above λ⋆, the base-relative extrapolated fixed point exits the clip-safe region defined by the clipped IS objective.The Bernoulli theorem provides the single-position result, with the base-neutral case as a special case.
  • Finite-budget behavior: Finite-budget dynamics can drift leftward with longer training, with the Fashion cliff midpoint moving 1.22 → 1.12 → 1.06 across N={42, 70, 200}.The paper supports boundary-seeking finite-budget dynamics but does not prove almost-sure convergence.
  • Sequence calibration: The sequence-level rule is provably safe below λ⋆(psafe, beff, c) under clipped IS, base-relative reverse-KL, and position-wise parametric reach.The multi-token lift is exact under off-modal-ratio invariance and approximate otherwise.
  • Scope and assumptions: The rule is scoped to near-deterministic, measurable scaffolds with parse headroom and sufficient finite-budget reachability.Outside that regime it may shift, abstain, or remain statistically underpowered.
  • Sequence calibration: Fashion calibration gives ptyp=0.9993 ± 0.0001, psafe≈0.99996, b≈0.81, and c=5, yielding a base-relative bracket [1.18, 1.28].The observed onset window [1.15, 1.25] falls within that bracket within one λ-grid step.

5 Experiments

The experiments test the clip-safety predicate across Fashion cliff localization, finite-budget extensions, scaling, controls, and scope checks. Results support sharp parse collapse near predicted thresholds, parameter-efficient ListOPD gains, and bounded generalization when theorem preconditions fail.

  • Experimental regime: The predicate requires near-deterministic structural tokens, one dominant binding class, SFT parse headroom, base-relative clipping, and sufficient training budget.Strict parse rate is the central dependent variable; ranking metrics are diagnostic.
  • Cliff localization: Parse transitions sharply between λ=1.20 and λ=1.25, while NDCG@1 on parsed outputs remains statistically flat across the sweep.Extending λ=1.15 training to 5 epochs reduces parse from 0.948 to 0.675, indicating finite-budget leftward drift.
  • Cliff localization: The 5-seed fine-grid sweep localizes the parse≥0.80 cliff to [1.204, 1.228], containing the predicted λ⋆=1.22.Seed variation increases approximately 4× across the boundary.
  • Finite-budget tests: The N=200 budget extension predicts a cliff midpoint of 1.061–1.068, with both estimates inside the locked [1.00, 1.10] bracket.Single-seed parses at λ∈{1.00,1.05,1.10} are {0.934,0.703,0.500}.
  • Parameter efficiency: Sub-threshold ListOPD reaches USEFUL∈[0.873,0.909] across 0.6B–8B students, while SFT is parse-limited at small and mid sizes.At λ=1.15, 1.7B-ListOPD reaches FMC=0.031±0.021; λ≥1.25 regresses to the 4B-SFT (K−1)-manifold.
  • Controls: Six pre-registered controls do not explain the 1.7B-SFT→ListOPD lift, and constrained decoding improves SFT parse without closing ListOPD’s USEFUL gap.Best constrained SFT reaches USEFUL=0.679 versus 0.874 for unconstrained ListOPD; permutation repair reaches 0.823.
  • Scope checks: The public tests refine scope: JSONSchemaBench lacks cliff localization under heterogeneous schemas, while a single outer K-ary class restores a bracketed but attenuated cliff.MS MARCO is reported as a boundary rather than a cliff replication because multi-seed results do not separate λ=1.25 from 1.5.
  • Cross-clip prediction: At c=1.5 and N=200, the predicted λ⋆typ=1.070 matches the observed midpoint 1.069 below grid resolution.The sub-critical anchor λ=0.95 remains at parse 0.943, while λ=1.20 collapses to 0.255.

6 Conclusion

The paper presents a closed-form, base-relative clip-safety threshold for reward-extrapolated OPD and validates it on structured-output ranking. Just below the threshold, ListOPD achieves parity with a larger SFT baseline, while the claims remain bounded by task, budget, and evaluator scope.

  • Conclusion: The predicate λ⋆(p,b,c) turns OPD tuning into a falsifiable boundary-prediction problem based on three measurable quantities.The paper frames the threshold as the point where extrapolation changes structured-output training from format-preserving to contract-collapsing.
  • Conclusion: Three pre-registered Fashion tests fall within locked prediction windows, including a small-clip prediction matched below grid resolution.Operating just below λ⋆ brings a 1.7B student to seed-noise parity with an 8B-SFT baseline at one-fifth the parameters.
  • Limitations: The predicate is base-relative and can shift, abstain, or lose power outside near-deterministic, parseable scaffolds or when finite budgets cannot reach the boundary.The parity claim uses Gemini-graded USEFUL, whereas the cliff diagnostic uses rubric-independent parse rate.
  • Practical rule: For finite-budget deployment, the paper recommends operating one λ-grid step below λ⋆(N) to retain margin for predicted leftward drift.ASPO exhibits the same cliff pattern one grid step earlier, supporting a mechanism-level interpretation rather than a method-specific one.

A Extended Discussion

The paper frames λ⋆ as a scope-sensitive operating rule whose evidence depends on matched clipping and near-deterministic structural-token conditions. In Fashion, ListOPD shows parameter-efficient in-domain parity, while constrained decoding and repair leave a measurable residual.

  • Scope and limitations: λ⋆ applies to near-deterministic structural tokens with parseable failures, and its operating rule is sensitive to the IS-clip implementation.The no-base variant does not show the cliff at the same 42-step budget.
  • Parameter efficiency: 1.7B-ListOPD matches 8B-SFT on Fashion in-domain at one-fifth the parameters, with USEFUL=0.857±0.016 versus 0.833±0.082.Parity is in-domain only; 8B-SFT retains an edge on Baby/Software.
  • Mechanism positioning: ASPO exhibits the same parse-collapse pattern one grid step earlier than vanilla OPD, positioning λ⋆ as a property of IS-clipped reverse-KL extrapolation.The comparison supports a mechanism-not-method interpretation across related methods.
  • Theory status and open directions: The theory proves the single-position fixed point, while the multi-token lift is exact only under off-modal-ratio invariance and approximate otherwise.Super-critical convergence is not established almost surely, and post-cliff parse-rate-drop magnitude lacks a closed-form prediction.
  • Deployment: Unconstrained ListOPD is the latency winner; XGrammar or llguidance adds +0.005 parse-rate insurance at <3% latency cost.Outlines is slower because its per-prompt finite-state machine is rebuilt over the review_id enum.
  • Constrained decoding and repair: +0.051 USEFUL remains against schema-constrained decoding plus permutation repair, so decoder-side repair alone does not close the ListOPD gap.Repair improves constrained-SFT outputs, but the residual supports a training-side contract-adherence claim rather than categorical superiority.

B.3 Reproducibility and data provenance

The paper reports a reproducibility and provenance boundary: its theorem-facing measurements are structurally verifiable, while Gemini-based relevance claims remain rubric-scoped. The sequence-level rule is conservative and conditional, with empirical cliff localization depending on calibration, equivalence-class assumptions, and finite-budget reachability.

  • Data provenance: Gemini 2.5 Pro supplies fixed scalar pseudo-labels for Amazon review-ranking, not human-validated relevance judgments.The paper does not claim that Gemini’s pretraining mixture is free of individual Amazon reviews or that its rubric transfers to human relevance.
  • Data provenance: The theorem-facing measurements are format validity, structural-token modal probability, and cliff location; NDCG and Kendall are rubric-side diagnostics.These measurements are separated from claims about absolute human relevance.
  • Sequence-level calibration: The sequence-level safety condition is sufficient but conservative because it requires every structural position to remain sub-critical.The empirical cliff instead reflects saturation by a Θ(1) fraction of effective token classes, typically far fewer than all structural positions.
  • Sequence-level calibration: The empirical sequence-level extension is conditional on a dense near-deterministic scaffold, SFT parse headroom, and finite-budget reachability of the boundary.The observed Fashion cliff is calibrated using safety and typical-class predictions rather than proved from first principles.
  • Implementation boundary: Finite-budget collapse depends on the implemented advantage and estimator, not solely on the algebraic clip-safe crossing.The base-relative implementation cliffs at λ∈[1.20, 1.25] in 42 steps, whereas the no-base patch remains parse-stable through λ=1.4.
  • Calibration results: At τ=0.9, the empirical-scale prediction is λ⋆(0.9993, 0.81, 5) = 1.28 and the safety bound is λ⋆(0.99996, 0.81, 5) = 1.18, bracketing onset [1.15, 1.25].The base-neutral special case gives λ⋆(0.9993, 1/2, 5) = 1.22, also within one λ-grid step.

C.2 Robustness to entropy-aware mixed objectives (EOPD)

The paper predicts that EOPD leaves the cliff threshold unchanged because its entropy gate is inactive at the binding structural positions. It does not analytically characterize how EOPD may alter finite-budget trajectories at non-binding positions.

  • The closed-form threshold is pinned by the most-concentrated structural position through a monotonicity argument.λ⋆ decreases with modal probability, so the binding position controls the boundary even when other positions activate the gate.
  • EOPD’s predicted λ⋆ coincides with OPD’s because the entropy gate is inactive at positions determining the threshold.The binding and typical structural classes have entropy upper bounds below τ, while marginal positions do not determine λ⋆.
  • The prediction concerns cliff position, not post-cliff dynamics or finite-budget first-passage time.EOPD may modulate non-binding trajectories and shift first passage, but those effects are not analytically characterized.
  • The fine-grid experiment evaluates whether the predicted boundary localizes across λ and seeds using parse-based cliff onset.The protocol uses a 5-seed Fashion sweep with λ values around the predicted threshold.
  • The 5-seed parse mean declines monotonically across the boundary, while NDCG@1 on parsed outputs remains statistically flat.Parse means change from 0.898 to 0.658, whereas NDCG@1 varies within 0.005.
  • The deeper parse cliff is localized near λ⋆=1.22, although the first detectable degradation begins slightly earlier.The parse≥0.80 and parse≥0.85 onset intervals contain λ⋆, while the parse≥0.90 interval lies about 0.01 below it.

D.2.1 Decisive ablation package

The ablation package tests whether ListOPD’s observed behavior reflects parse recovery, teacher scale, training budget, decoding, or seed variability. Across these checks, parse adherence is the main changing component, while several alternative explanations are constrained.

  • Fashion-trained ListOPD recovers cross-category USEFUL primarily through parse-rate recovery, not large changes in NDCG@1 on parsed outputs.At 1.7B, USEFUL rises to 0.707 on Baby Products and 0.749 on Software with parse rates above 91%.
  • 1.7B-OPD at λ=1.15 reaches FMC=0.031 ± 0.021, matching the 8B-SFT failure regime rather than the 4B-SFT K−1 regime.Self-distillation and continued SFT controls address on-policy exposure and extra-steps confounds.
  • A 32B teacher yields stable USEFUL=0.899–0.909 for an 8B student across the tested λ band, supporting scale feasibility rather than a new cliff calibration.The spot-check peaks at λ=1.22 but is explicitly not interpreted as categorical teacher superiority.
  • Extending training from 42 to 70 steps shifts the λ=1.15 parse rate from 0.948 to 0.675, demonstrating a budget-dependent leftward cliff shift.λ=1.10 remains stable, while λ=1.20 and 1.25 collapse further under the longer budget.
  • Sampling does not dissolve the cliff: temperature changes parse by at most 7 points and never raises λ=1.25 above 0.7.The preregistered temperature ablation passes its locked thresholds.
  • Boundary variance expands sharply, with cross-seed parse standard deviation rising from 0.016 to 0.098–0.101 across the pilot λ range.This matches the predicted diffusion-dominated behavior after trajectories leave the clip-safe region.

E.2 Pre-registered budget-N test of Thm. 4.2

The budget-extension tests support a finite-budget interpretation of the cliff: cumulative drift can reach the clip-unsafe regime as training time increases, while clip strength changes reachability rather than simply ordering outcomes asymptotically.

  • The single-seed cliff midpoint is 1.061, inside the locked bracket and consistent with the central 1/N prediction of 1.023.The midpoint is interpolated between parse 0.703 at λ=1.05 and 0.500 at λ=1.10.
  • Intermediate checkpoints recover the predicted first-passage pattern: higher λ crosses the parse-safe band earlier, while λ=1.00 remains safe.At λ=1.05, the trajectory crosses the 0.90 band between steps 80 and 120 and ends at parse 0.703.
  • The final-step teacher/student ratio rises from approximately 9 at λ=1.0 to 30.9 at λ=1.4, then collapses to approximately 5 post-cliff.This pattern is interpreted as boundary-seeking flow followed by a degenerate terminal regime.
  • At fixed λ=1.15 and N=42, peak ratio is non-monotone across c={1.5,2,5,∞}, showing finite-budget reachability rather than asymptotic log-c ordering.The c=1.5 cliff appears only when the budget reaches N=200.
  • The third budget point extends the original finite-budget test by approximately 4.7× and quantitatively supports leftward cliff drift.The preregistered grid, success criterion, and failure taxonomy were locked before observation.
  • The prediction applies within the K-ary listwise SFT regime and does not establish scope outside that setting.The aggregator and scaffold assumptions remain explicit boundaries of the theorem’s predictive scope.
  • The c=1.5, N=200 test matches the predicted midpoint: 1.0695 versus λ⋆typ(c=1.5)=1.070.The sub-critical λ=0.95 anchor remains at parse 0.943, while λ=1.20 collapses to 0.255.

F.3 Public-benchmark replication on JSONSchemaBench: lift transfers, cliff scope-bounds to K-ary listwise

JSONSchemaBench separates transfer of deployment utility from localization of a sharp cliff. The single-instance heterogeneous-schema setup does not localize a cliff, whereas a K=4 outer array restores the binding-class condition and yields a predicted but shallower boundary.

  • On single-instance JSONSchemaBench, no cliff localizes on the locked λ-grid, despite OPD validation exceeding the 4B-SFT teacher baseline.Validate rate remains 0.60–0.635 across λ=1.10–1.45, including 0.635 at λ=1.45 versus the teacher’s 0.625.
  • The deployment-useful lift transfers: 1.7B-SFT validate 0.535 rises to 1.7B-OPD validate 0.60–0.635, matching the 4B-SFT teacher.This supports parameter efficiency on a public benchmark while leaving cliff localization unresolved.
  • The single-instance null is attributed to heterogeneous schemas violating the single-binding-class precondition, not to warmstart base misspecification.Worst-case b choices would still permit grid localization if the structural precondition held.
  • The K=4 cliff midpoint is 1.29 within the predicted [1.19,1.42] bracket, but the super-critical regime stabilizes near klist≈0.33 rather than collapsing to zero.The result is therefore a location match with attenuated collapse sharpness.
  • The K=4 outer array restores the binding class through the outer K−1→K closing transition and recovers a cliff in the predicted bracket.Inner item contents may remain heterogeneous while the outer transition supplies the dominant class.
  • Uniform inner schemas distinguish Fashion’s sharp collapse from the shallower K=4 replication.The K=4 OPD lift is +0.04 klist_rate versus Fashion’s +0.32, and a default-valued K-array attractor appears at high λ.
  • The function-calling benchmark provides a scale check rather than an independent closed-form calibration because code-specific peff was not measured.Parse and USEFUL peak near λ=1.15–1.25 and decline at λ=1.4, more gently than Fashion.
  • Function-calling scope is limited by SFT parse saturation and a capability ceiling on parallel categories.SFT-4B already reaches macro parse 0.942, while AST match for parallel subsets is at or below 0.005.

G ASPO head-to-head: same-mechanism fix preserves the cliff

The ASPO ratio-flip fix improves stability at λ=1.0 but preserves the same sharp cliff, with no categorical evidence that ListOPD dominates ASPO.

  • Same-mechanism comparison: At λ=1.0, ASPO improves over vanilla OPD by about +4.4pp in parse and USEFUL.ASPO reaches 0.932±0.008 parse and 0.863±0.006 USEFUL, versus 0.887 parse and 0.819 USEFUL for vanilla OPD.
  • Same-mechanism comparison: ASPO λ=1.0 reaches USEFUL=0.863±0.006, comparable to 5-seed ListOPD λ=1.15 at 0.857±0.016.The single-seed ListOPD headline of 0.882 is not used to claim superiority over ASPO.
  • Cliff preservation: At λ=1.5, ASPO parse drops to 0.096±0.020 from 0.932±0.008 at λ=1.0, and remains collapsed at λ=2.0 and λ=3.0.Its cliff begins one grid step earlier than vanilla OPD, which collapses between λ=1.25 and λ=1.40.
  • Cliff preservation: The shared cliff pattern under ASPO supports interpreting the transition as a mechanism-level effect rather than a narrow ListOPD implementation artifact.ASPO addresses the same positive-advantage-token IS asymmetry with a ratio-flip fix.

H Predicate λ⋆(p, c) scope tests

The scope tests distinguish the algebraic clip-safe crossing from finite-budget collapse and identify settings where the predicate's assumptions or classifier are insufficient.

  • Scope and failure modes: Table 31 separates calibrated regimes, public stress tests, precondition failures, and failures of the finite-budget classifier.The predicate can locate an asymptotic fixed point without guaranteeing finite-N first-passage time under altered estimators.
  • Scope and failure modes: The c=1.5 test is an explicit finite-budget failure: the fixed point exceeds the clip-safe boundary, but the 42-step run remains parse-stable.This separates asymptotic boundary crossing from observed collapse within a fixed training budget.
  • Uncompleted tests: Two pre-registered ablations remain deferred because shared-cluster multi-tenancy prevents stable concurrent training launches at the paper's configuration.The blocked extended regularizer sweep and matched cross-architecture extension are described as operationally, not methodologically, blocked.

I.1 Regularizer protocol, predicted shifts, and pilot results

The regularizer analysis predicts small threshold shifts at the Fashion operating point and tests them with pre-registered pilots, while additional configurations remain budget-blocked.

  • Predicted shifts: Adding an entropy bonus contributes γ ∂θH = −γ logit(q) q(1−q) to the expected θ-flow under standard SGD.The q(1−q) factor is the parametric-loss chain-rule Jacobian used by verl.
  • Predicted shifts: At Fashion, the entropy-induced shift is δλ ≈2.1×10^-4 γ base-relative or ≈1.7×10^-4 γ with base-neutral b=1/2.With ptyp=0.9993 and c=5, qc(1−qc)≈1.4×10^-4 suppresses the slope.
  • Predicted shifts: The entropy bonus does not perceptibly shift the Fashion cliff to four decimals across γ within the deployable range.The closed form is linear in γ for the stated regularizer parameterization.
  • Pilot observations: At λ=1.15, pilot parse fell from 0.921±0.019 to 0.887 with KL-to-base and 0.731 with entropy, while parsed-subset NDCG@1 remained near baseline.The pilots used β=0.01 or γ=0.001, three epochs, one seed, and 42 optimizer steps.
  • Protocol and scope: The pre-registered protocol covers β∈{0.05,0.20} across λ∈{1.20,1.30,1.40}, with strict parse rate as the primary observation.The full regularizer sweep required 18 runs and was feasible within an overnight single-node budget absent the multi-tenancy issue.
Loading 2605.08737v1…