PaperlayerView exact revision on arXiv

Source-linked AI summary

Exact Feedback Is Not Control: Evaluating Text-based Closed-Loop Revision in LLMs

Haitong Jiang, Chunlin Liu, Yile Wang, Yuhong Feng

arXiv:2609.28150v1cs.CL

TL;DR

The paper asks whether poor closed-loop revision reflects unreliable feedback or models’ difficulty acting on correct feedback. It evaluates fixed-budget revision with deterministic verifiers and finds persistent model-specific differences, while exact feedback makes errors observable without making revision reliable.

  • Problem

    When feedback may be unreliable, poor revision outcomes leave unresolved whether the bottleneck lies in feedback or in models’ responses to it.

    Source: §1 · p. 1

  • Method

    A fixed-budget revision framework uses deterministic verifiers to report all remaining violations and apply the same frozen rules to final success.

    Source: §1 · p. 2

  • Results

    Substantial cross-model and cross-constraint revision gaps persist from identical starting drafts, alongside reproducible model-specific feedback-to-action patterns.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Exact feedback provides observability, not control; reliable revision requires effective model responses as the revision state evolves.

    Source: §6 · p. 9

    The fixed-budget, deterministically checkable text tasks do not cover subjective objectives, incomplete feedback, or changing environments.

    Source: §6 · p. 9

Abstract

from arXiv · show

Closed-loop revision is increasingly used in large language model (LLM) applications, but failures may reflect incomplete feedback or ineffective responses to correct feedback. We introduce a fixed-budget revision protocol with deterministic verifiers that report all remaining violations across exact-length, lexical, and compositional constraints. Fixing feedback correctness and completeness isolates model-side revision behavior. Across 19 open- and closed-source models, controller-level mean final joint success ranges from 17.4% to 99.8%, with substantial cross-model gaps persisting under identical initial drafts. Controlled experiments reveal reproducible model-specific responses to exact feedback. Post-training and scale reshape these responses without consistently bringing them closer to exact correction. Across all constraint families, failed trajectories often repeat earlier outputs, and prior recurrence is associated with lower subsequent recoverability. Matched-state interventions show that removing earlier dialogue while holding the current draft and feedback fixed changes recurrence escape without reliably improving final success; effects depend on the model, task, and trigger-state composition. Exact feedback makes revision errors observable, but does not make the closed loop reliable. Code and reproduction instructions: https://github.com/kevinjiang0121-cyber/exact-feedback-code.

1 INTRODUCTION

The paper isolates model-side closed-loop revision by supplying complete, exact feedback, then studies how models respond across constraints and repeated revisions. It finds persistent model-specific differences, including in feedback-to-action patterns and recurrent failure trajectories.

1 INTRODUCTION

  • When feedback may be unreliable, poor revision outcomes do not reveal whether the bottleneck is feedback or the model’s response to it.The paper asks how reliably models act on complete and correct feedback over repeated revisions.Source: §1 · p. 1
  • The framework evaluates fixed-budget closed-loop revision under explicit, deterministically checkable constraints.The authors use verifiers that report all remaining violations after each revision.Source: §1 · pp. 1–2
  • The evaluation suite uses a fixed-budget request–model–verifier loop with frozen rules that report all violations and determine final success.It includes frozen cases, trajectory-level records, and generation and analysis pipelines for reproducible evaluation.Source: §1 · p. 2
  • With identical starting drafts, substantial revision gaps persist across 19 model configurations and three constraint families.These gaps are not simply inherited from differences in initial generation.Source: §1 · p. 2
  • Exact feedback does not make revision reliable: reproducible, model-specific feedback-to-action patterns persist across controlled correction requests and inform independent multi-round revision.Post-training and model scale reshape these behaviors in different ways.Source: §1 · p. 2
  • Failed revision trajectories frequently revisit earlier outputs across all three constraint families, and matched-history interventions alter recurrence escape and subsequent editing behavior.The interventions produce much smaller and heterogeneous changes in final success.Source: §1 · p. 2

2 RELATED WORK

Prior work studies iterative refinement, exact verification, and trajectory behavior, but leaves open how models act on the same draft and complete feedback over repeated revisions. This paper positions controlled correction requests and matched-state history interventions as ways to investigate those questions.

Iterative refinement and self-correction

  • Self-Refine, CRITIC, and SCoRe improve revision through different feedback sources or changes to revision policy, but effectiveness depends on setting.Studies report that self-correction without external feedback can fail to improve answers and that feedback quality limits code repair.Source: §2 · p. 2
  • Prior refinement studies leave open whether cross-model gaps persist when starting drafts and complete, exact feedback are held fixed.The open question also includes whether gaps reflect reproducible differences in translating feedback into revisions.Source: §2 · p. 3

Exact verification and constrained revision

  • Constrained decoding and length-control methods enforce output requirements, while this work studies revision under exact feedback with standard decoding.Related work also examines sound external feedback and critique-guided revision.Source: §2 · p. 3

Closed-loop behavior and history

  • Earlier trajectory studies track repeated answers, correctness transitions, or textual fixed points; this work tests history after requirements are fully specified.Holding the current draft and verifier report fixed separates earlier-dialogue effects from differences in draft or feedback, with next-revision behavior and final success measured separately (Wu et al., 2026).Source: §2 · p. 3

3 A UNIFIED CLOSED-LOOP REVISION PROTOCOL

The protocol evaluates whether an LLM can satisfy every verifiable constraint in a shared draft within a fixed revision budget. Deterministic verifiers report remaining violations, while the same frozen rules define success across three text-based constraint families.

Problem definition

  • The protocol studies whether a model can satisfy all constraints in one draft within a budget of T revisions.Joint success requires every constraint to hold in the same draft.Source: §3 · p. 3

Revision loop

  • Each configured model revises the current draft using the task, verifier report, and retained history; the verifier checks success and reports remaining violations.The configured model is termed a “controller,” or a “reviser” when its role is revising a supplied draft.Source: §3 · p. 3
  • The verifier reports all remaining violations, and the same frozen rules generate feedback and determine final success.This makes feedback correctness and completeness controlled experimental conditions for isolating model-side revision.Source: §3 · p. 3

Constraint families

  • The three constraint families are exact-length, lexical, and compositional, with fixed rules for counting, sentence splitting, and matching.Exact-length tasks combine a requested word count with required content anchors; compositional tasks combine sentence count with forbidden words or per-sentence length bounds.Source: §3 · pp. 3–4, Figure 2 · p. 4

Loop outcomes

  • Capture time records the first draft that satisfies all constraints; the loop stops at success or when the revision budget is exhausted.Final success indicates whether capture time occurs within the budget.Source: §3 · p. 4

Evaluation design

  • The main experiments use eight revisions and evaluate 480 cases per constraint family across 19 model configurations, yielding 27,360 attempted trajectories.Each model receives the same maximum number of revisions, though token costs can differ; the paper notes that eight rounds do not imply convergence.Source: §3 · p. 4

4 REVISION PERFORMANCE AND FEEDBACK-TO-ACTION RESPONSES

Final joint success varies widely across models and constraint families, and recovery profiles differ over revision rounds. Controlled tests also find reproducible, model-specific responses to exact feedback that post-training and scale reshape without consistently improving exact correction.

4.1 REVISION PERFORMANCE ACROSS ROUNDS AND MODELS

  • Recovery timing differs by task: exact-length success can rise late, lexical gains mostly fade after early rounds, and compositional success is often front-loaded.Some controllers continue making meaningful gains near the end of the eight-revision budget, while others plateau earlier.Source: §4.1 · p. 5, Figure 3 · p. 5
  • Controller-level mean final joint success ranges from 17.4% to 99.8%, with compositional tasks generally outperforming exact-length and lexical tasks.Table 1 reports final joint success across the three constraint families.Source: §4.1 · p. 4, Table 1 · p. 5
  • Even from identical exact-length starting drafts, Llama 3.1 8B retains a 42.5–77.1 percentage-point final-success advantage across six significant reviser comparisons.The advantage remains 45.3–77.1 points after initially successful drafts are excluded.Source: Table 2 · p. 5, §4.1 · p. 6

4.2 REVISION RESPONSES ACROSS POST-TRAINING AND MODEL SCALE

  • Controlled one-step revisions vary the target word count and matching verifier feedback on fixed source drafts, then test whether response curves generalize to natural revision states.Discovery curves are estimated on 24 texts and evaluated on 48 disjoint confirmation texts.Source: §4.2 · p. 6
  • Post-training reshapes Base-model revision behavior differently for Llama and Qwen, while increasing Qwen3 scale does not consistently move responses toward exact correction.Llama instruction tuning largely removes a deletion-side dead zone; Qwen instruction tuning reduces error-expanding edits but increases zero-action responses. Qwen3 32B can over-correct deletions while under-correcting additions, whereas Gemma 2 9B tracks exact correction relatively closely.Source: §4.2 · pp. 6–7
  • Model-specific response curves replicate across unseen texts and reduce edit-prediction MAE on independent multi-round trajectories by 1.91 words (11.0% relative; 95% CI 8.5–13.9).The curves correlate between discovery and confirmation texts at Spearman ρ > 0.85 for every checkpoint.Source: §4.2 · pp. 6–7

5 FAILURE DYNAMICS AND TARGETED INTERVENTIONS

Failed trajectories often repeat complete outputs across all three constraint families, and prior recurrence is associated with lower later recoverability. Matched-state history removal substantially changes continuation behavior, but final-success effects are smaller and heterogeneous.

5.1 OUTPUT RECURRENCE ACROSS CONSTRAINTS

  • Recurrence occurs in 84.8% of 5,272 exact-length, 78.7% of 4,805 lexical, and 90.2% of 2,093 compositional failed trajectories.Figure 5 classifies recurrence patterns among failures, normalizing each model’s bars by its failure count within the constraint family.Source: §5.1 · p. 7, Figure 5 · p. 7
  • Terminal fixed points or cycles account for 69.0%, 57.6%, and 79.5% of failures in exact-length, lexical, and compositional tasks, respectively.Fixed points alone account for 55.4%, 48.9%, and 71.2%; rates are conditional on failure, and repetition is not a complete explanation of model quality.Source: §5.1 · p. 7
  • Adjusted odds ratios for later success comparing repeat fractions 1 versus 0 are 0.059 locally, 0.017 for API models, and 0.047 combined.This negative association concerns cases unresolved after revision 4 and is adjusted for measured revision state and history; it does not show that repetition causes failure.Source: §5.1 · p. 8, Table 3 · p. 8

5.2 HISTORY INTERVENTIONS AT MATCHED STATES

  • Removing earlier history raises next-step escape by 42.2 points for Llama and 49.2 for GLM, but final success rises only 10.8 and 5.0 points.Final success reaches 21.7% for Llama and 8.3% for GLM; most reset continuations still fail within the remaining budget.Source: §5.2 · p. 8, Figure 6 · p. 8
  • Across 868 matched recurrent states, reset effects on escape range from −60.6 to +36.0 points across nine model–task cells, while final-success changes are smaller in eight.The remaining Falcon sentence-length cell has only 11 states and is descriptive; excluding first-revision repeats reverses some escape effects.Source: §5.2 · p. 9, Figure 7 · p. 9
  • Matched-state experiments show that changing retained dialogue alters continuation behavior with the current draft and feedback fixed, but history manipulation alone does not make revision reliable.Effects depend on model, task, and trigger-state composition.Source: §5.2 · pp. 8–9

6 DISCUSSION AND CONCLUSION

With feedback correctness and completeness fixed, closed-loop revision still depends on the model’s interaction with an evolving revision state. The discussion and appendices define the evaluation’s scope, verifier conventions, data, and controller panel.

6 DISCUSSION AND CONCLUSION

  • Exact feedback exposes revision errors but does not ensure reliable completion; recurrence marks states with lower subsequent recoverability, while history changes local behavior without reliably improving final success.The same history intervention can help or hinder recurrence escape across models and tasks.Source: §6 · p. 9
  • The fixed-budget, deterministically checkable text tasks do not cover subjective objectives, incomplete feedback, or changing environments.Within this scope, evaluation should include feedback incorporation, recurrence, and retained history, not only endpoint success or single-step improvement.Source: §6 · p. 9

A.1 VERIFIERS AND VIOLATION MEASURES BY CONSTRAINT FAMILY

  • The verifier uses frozen tokenization and sentence splitting, and each case has a human-authored witness that passes the same verifier.The appendix defines joint success, violation vectors, scalar violation summaries, and trajectory events including recurrence, fixed points, cycles, and escape.Source: §A.1 · pp. 15–16

5. Near target (≤

  • The appendix’s exact-length failure categories partition failures by a fixed priority rule, although their underlying conditions can overlap.The roster spans seven model families and dense, MoE, and Transformer–SSM hybrid architectures.Source: §5 · p. 16, §A.3 · p. 17

A.2 DATA CONSTRUCTION AND FROZEN EXECUTION

  • The exact-length panel contains 480 cases from four sources, with targets spanning 10–500 words and 1,907 frozen anchors.The lexical and compositional panels each contain 480 frozen COLLIE-derived cases, covering three lexical structures and two compositional structures.Source: §A.2 · pp. 16–17
  • Each task, target, prompt, revision budget, and stopping rule is fixed before outcomes; the loop provides full verifier reports and stops at success or after eight revisions.The evaluation includes 12 locally executed checkpoints and seven API controllers; one Gemini refusal leaves 3,359 protocol-complete API exact-length records and 479 cases common to all seven API controllers.Source: §A.2 · p. 17, §A.3 · p. 17

A.4 REVISION-BUDGET SENSITIVITY

Revision-budget sensitivity and controller-by-constraint interactions reveal heterogeneous outcomes across settings. Longer budgets preserve ordering reversals, while late recovery and controller rankings vary by task and model.

A.4 REVISION-BUDGET SENSITIVITY

  • Cross-family controller ordering reversals persist at every inspected revision cap, although individual rankings change.Absolute success changes with the action budget.Source: §A.4 · p. 17
  • 12 of 125 context-feasible cases from a selected set of 144 revision-8 failures succeed by revision 32; this cohort does not estimate population-wide gains.113 remain unsuccessful and 19 exhaust the native context window.Source: §A.4 · p. 18

A.5 EVIDENCE PANELS

  • Tables 5–7 report cumulative success at each revision cap, showing where rates plateau or continue rising late in the budget.The evidence-panel table specifies each panel’s scope and evaluation target.Source: §A.4 · p. 18, Table 8 · p. 19

A.4 REVISION-BUDGET SENSITIVITY

  • From T = 4 to T = 8, DeepSeek V4 Flash gains 29.0 points, GLM-5.2 gains 26.3, and Llama 3.1 8B gains 22.9 on exact length.The largest compositional gain over the same interval is 4.0 points for Falcon-H1 7B; lexical cells generally flatten earlier, though Gemini 3.6 Flash gains 11.9 points.Source: Table 7 · p. 19

B.1 INTERACTION BETWEEN CONTROLLER AND CONSTRAINT FAMILY

  • Llama 3.1 70B minus Qwen3.7 Plus changes from +45.1 points on exact length to −22.5 on lexical constraints, a −67.6-point difference-in-differences.After Holm correction, 19 of 21 exact-length pairs and 20 of 21 lexical pairs differ in final success; compositional constraints yield 9 of 21 pairwise separations.Source: §B.1 · p. 20
  • Controller-by-family interaction is significant, χ2(12) = 1214.05 (p = 1.63×10^-252), with controller-by-structure interaction χ2(16) = 3662.21 (p < 10^-300).These estimates compare cells across distinct datasets and COLLIE case sets.Source: §B.1 · p. 20

B.2 CONTENT CONTRACT ROBUSTNESS CHECK (DESCRIPTIVE)

The content-contract check finds that Llama remains ahead under stricter measured requirements, while prompt and decoding variations change absolute rates without changing the tested controller ordering. Execution records also delimit reproducibility.

B.2 CONTENT CONTRACT ROBUSTNESS CHECK (DESCRIPTIVE)

  • Llama remains ahead under every measured content condition, including a 54.0-point gap under full applicable-entity retention.The checks measure content retention, not semantic equivalence or subjective quality.Source: §B.2 · p. 20

B.3 INFERENCE AND EXECUTION PROVENANCE

  • A guarded batch-four Llama trial failed exact serial equivalence on seven of eight gate cases and was excluded from the canonical output.The batch-one gate matched every case, round, and generated text.Source: §B.3 · p. 21
  • Launch records omit driver versions, full weight hashes, and a single immutable Qwen3 14B revision, limiting exact provenance for those fields.The manifests record input paths, row counts, and SHA-256 digests; the missing fields are marked as missing.Source: §B.3 · p. 21

B.4 PROMPT AND DECODING ROBUSTNESS

  • Across every tested feedback prompt form and sampled decoding seed, Llama ranks above Gemma, which ranks above Qwen3 14B; absolute rates vary with the interface.The comparison uses the same 60 case IDs for all three models.Source: §B.4 · p. 21

C.1 RESPONSE ESTIMATION, PREDICTION, AND RECOVERABILITY

Across controlled and natural revision analyses, model-specific response patterns replicate, while recurrence is associated with lower later recoverability. Matched-history and weight interventions alter revision behavior, but their effects vary by model and task and do not reliably improve final success.

C.1 RESPONSE ESTIMATION, PREDICTION, AND RECOVERABILITY

  • Under the controlled, text-balanced protocol, Qwen3 30B-A3B showed 59.5% zero word-count change, 35.0% correct direction, and 26.9% error contraction on confirmation texts.On confirmation texts, Qwen instruction tuning reduced error expansion from 47.3% to 14.2% while increasing zero action from 35.9% to 49.8%; these are descriptive checkpoint contrasts, not an isolation of training from other checkpoint differences.Source: §C.1 · p. 22
  • 17.45 versus 15.54 words: model-specific response curves reduce held-out edit-prediction MAE by 1.91 words (95% CI 1.58–2.25), or 11.0% (95% CI 8.5–13.9), relative to the shared curve.The aggregate averages across nine checkpoints on independent natural trajectories.Source: §C.1 · p. 23, Figure 8 · p. 23, Table 14 · p. 23
  • Recurrence by revision 4 is negatively associated with later recovery across 11 of 12 local checkpoints and all four sources, with negative adjusted API associations at adjacent landmarks.The Qwen3 1.7B ceiling-limited exception has only four unexposed cases.Source: §C.1 · p. 24, Figure 9 · p. 24

C.2 STATE-MATCHED HISTORY INTERVENTION

  • Resetting history raised Llama final joint success from 10.8% to 21.7% and GLM success from 3.3% to 8.3%, yet most reset continuations still failed.The matched intervention varied retained dialogue while holding the current state fixed; escape and final success were distinct outcomes.Source: §C.2 · pp. 24–25

C.3 CROSS-CONSTRAINT HISTORY EFFECTS

  • History-reset effects differ by model and task: Gemma escape rose on word-position and sentence-length tasks but fell on sentence-final-word tasks, while Qwen sentence-length success changed by 0.6 points (95% CI −2.8 to 3.9).Excluding first-revision repeat states reverses some escape effects, including Qwen on sentence-length tasks.Source: §C.3 · p. 25

C.4 PARTIAL-HISTORY MATCHED INTERVENTION

  • The GLM partial-history pilot compares full history with last-two-pair, last-pair, and current-state-only conditions on 60 recurrent states, using escape as the primary endpoint.Contraction, later recurrence, and final success are secondary endpoints; the supplied passages give the design but not outcome estimates.Source: §C.4 · p. 26

D ALIGNED CHECKPOINT LOCALIZATION

  • Aligned checkpoint probes read error sign and required-correction magnitude, but readable representations do not establish that generation uses the decoded information.In Llama’s descriptive layer sweep, correction-magnitude R2 rose from 0.08 at layer 32 to 0.62 at layer 14 for Base, and from 0.44 at layer 32 to 0.93 at layer 8 for Instruct; these selected-layer values are upward-biased.Source: D ALIGNED CHECKPOINT LOCALIZATION · pp. 26–27, §D.1 · p. 27, §D.3 · p. 28

D.4 LLAMA PARAMETER RESTORATION

  • Weight and activation interventions changed local revision behavior without reliably transferring effective correction: Llama transplantation improved direction but worsened calibration, and Qwen restoration or graft conditions failed the specified transfer criteria.The Llama joint graft improved direction but missed the calibration gate; Qwen restoration increased error expansion on held-out fixed states, while its tested activation and graft conditions did not pass criteria.Source: §D.1 · p. 27, §D.5 · p. 29, §D.6 · p. 29

D.8 GEMMA: DIFFERENT EFFECTS ON REPETITION AND CORRECTION

In Gemma, Base-weight restoration produced distinct forms of poor revision rather than a uniform behavioral change. A tested Instruct graft improved direction in discovery, but did not establish confirmed recovery.

D.8 GEMMA: DIFFERENT EFFECTS ON REPETITION AND CORRECTION

  • Restoring Gemma blocks 21–30 lowered correct direction by 65.6 points, while restoring blocks 31–41 increased error expansion by 33.9 points on held-out fixed states.Across closed-loop cases, blocks 21–30 increased recurrence by 23.1 points and lowered success by 29.8 points; blocks 31–41 reduced recurrence by 38.5 points while lowering success by 22.9 points and increasing terminal absolute error by 118.8 words.Source: §D.8 · p. 30, Figure 13 · p. 30, Table 20 · p. 30
  • Grafting Instruct blocks 21–30 plus output into Base raised fixed-state direction accuracy from 51.0% to 74.0% and discovery-trajectory success from 1/120 to 11/120, but calibration remained below the control.The result is discovery-only partial transfer: graft confirmation is incomplete and excluded from reported estimates and conclusions.Source: §D.8 · pp. 30–31, Table 21 · p. 31

E ALIGNED POST-TRAINING AND INTERFACE COMPARISONS

Aligned Llama–Tülu checkpoints vary substantially across task structures, and native formatting changes outcomes differently by checkpoint and constraint family. Neither the common-interface nor native-template comparison isolates the causal effect of a training stage.

E ALIGNED POST-TRAINING AND INTERFACE COMPARISONS

  • Post-training outcomes vary by task structure: SFT reached 55.4% on c09 but 2.5% on c10, while DPO and RLVR rankings also differed across structures.Checkpoint-by-structure interaction was χ2(8) = 1094.58, p = 5.66 × 10−231.Source: E ALIGNED POST-TRAINING AND INTERFACE… · p. 31
  • Native formatting changed the DPO–RLVR ordering: RLVR led by 3.1 points under the common interface, but DPO led by 15.8 under native serialization, a 19.0-point paired difference-in-differences (95% bootstrap CI 12.9–25.2).Interface effects also varied across task structures and checkpoints; neither comparison isolates a training-stage causal effect.Source: E ALIGNED POST-TRAINING AND INTERFACE… · p. 31, Table 22 · p. 31

F.1 FIXED-DRAFT CROSSOVER

The fixed-draft crossover tests revisers on shared starting outputs, while supplementary audits describe capture under the revision budget and sensitivity to first-revision trigger states. These analyses identify both persistent controller differences and boundaries on interpreting history-reset contrasts.

F.1 FIXED-DRAFT CROSSOVER

  • Across shared drafts, the conditional reviser advantage after excluding initially successful drafts ranges from 45.3 to 77.1 percentage points, while no draft-source contrast survives Holm correction in the all-case panel.The crossover includes 2,400 audited trajectories and compares Llama with peer revisers on common cases.Source: §F.1 · p. 32, Table 23 · p. 32

F.2 CAPTURE AND BUDGET

  • Among 650 API failures, 90.2% ended at their own best absolute residual, while conditional capture averaged over revisions 5–6 ranged from 6.4% to 58.7% across systems.Late per-round capture was no lower than early capture for every system; the residual diagnostic does not rule out intermediate oscillation or instability.Source: §F.2 · p. 32, Figure 14 · p. 33, Figure 15 · p. 33

F.3 TRIGGER-STATE SENSITIVITY OF HISTORY RESET

  • All 209 first-revision reset states replayed the previously failing prompt, and 199 repeated the trigger text, contributing to mixed-cohort history-reset contrasts.Removing these states changes the analyzed population rather than estimating a causal effect of intervention timing.Source: §F.3 · p. 33

G FAILURE TRAJECTORIES ACROSS CONSTRAINT FAMILIES

The section classifies failed trajectories across exact-length, lexical, and compositional constraints by their output-repetition patterns. It reports failure counts across all complete records and shows within-family failure compositions across 19 model configurations.

G FAILURE TRAJECTORIES ACROSS CONSTRAINT FAMILIES

  • Of 27,359 complete trajectories, 12,170 failed: 5,272 exact-length, 4,805 lexical, and 2,093 compositional; one protocol-incomplete Gemini exact-length record was excluded.Counts use all complete records, not only the 479 cases shared for paired endpoint comparisons.Source: G FAILURE TRAJECTORIES ACROSS… · pp. 33–34
  • Failed trajectories are classified in priority order as terminal fixed points, terminal cycles, other repeats, or no repeats, using exact UTF-8 output equality.Fixed points require three identical ending drafts; tested cycles have repeated blocks of period 2–4, while other repeats are nonterminal or untested patterns.Source: G FAILURE TRAJECTORIES ACROSS… · p. 34
  • These mutually exclusive categories describe output trajectories, not causal mechanisms, and their labels are not interchangeable with Appendix A.1’s seven exact-length categories.The two schemes use different criteria, including word-count thresholds, anchor status, and signed-error flips in Appendix A.1.Source: G FAILURE TRAJECTORIES ACROSS… · p. 34
  • Figures 16–18 show within-family failure compositions for exact-length, lexical, and compositional constraints across all 19 models.Their panel (a) compositions pool failures within API, local, or all models; models with more failures contribute more, so these are not equally weighted model averages.Source: G FAILURE TRAJECTORIES ACROSS… · p. 34, Figure 16 · p. 35, Figure 17 · p. 35

Related papers

Loading 2609.28150v1…