PaperlayerView exact revision on arXiv

Source-linked AI summary

Before the Rollout Ends: Early Terminal Reward Prediction for Long-horizon Coding Agents

Jihan Yao, Sihan Zeng, Shangbin Feng, Zhiyuan Fan, Banghua Zhu, Yulia Tsvetkov

arXiv:2609.31995v1cs.CL

TL;DR

Long-horizon coding agents receive verifiable rewards only after expensive sequences of tool calls, delaying success evidence and leaving terminal feedback sparse. CER predicts terminal reward from visible prefix behavior using shared, task- and stage-specific rubrics informed by group comparisons and historical experience. CER improves TTS RM@8 over strongest baselines by 4.2 pp on Nemotron 3 Ultra and 2.0 pp on Qwen 3.6 27B; in three-fold Qwen 3.5 9B RL, it exceeds complete-rollout TMax (51.6% vs. 49.7%) using 52.7% fewer online tokens.

  • Problem

    Long-horizon coding agents receive verifiable rewards only after expensive sequences of tool calls, delaying success evidence and leaving terminal feedback sparse.

    Source: §1 · p. 1

  • Method

    CER predicts terminal reward from visible prefix behavior using shared, task- and stage-specific rubrics informed by group comparisons and historical experience.

    Source: §1 · p. 1, §3.2 · p. 4

  • Results

    CER improves TTS RM@8 over strongest baselines by 4.2 pp on Nemotron 3 Ultra and 2.0 pp on Qwen 3.6 27B; in three-fold Qwen 3.5 9B RL, it exceeds complete-rollout TMax (51.6% vs. 49.7%) using 52.7% fewer online tokens.

    Source: §1 · p. 2

  • Takeaways & Limitations

    CER supports efficient test-time scaling and verifier-free online RL as an interpretable, dense evaluation method for long-horizon coding agents.

    Source: §7 · p. 9

    The RL comparison is data-limited: TMax uses 14,600 synthetic tasks, while each SWE-bench fold trains on 250 tasks.

    Source: §5.2 · p. 8

Abstract

from arXiv · show

Long-horizon coding agents receive verifiable rewards only after completing expensive sequences of tool calls. This increases inference cost, amplifies early wrong hypotheses, and can lead to sparse terminal reward and unstable training. We introduce Contextual Early Reward (CER), which predicts terminal reward through behavioral evidence in a trajectory prefix. CER synthesizes adaptive rubrics specific to the current task and stage through experiences summarized from related historical tasks. In test-time scaling on SWE-bench Verified, CER improves RM@8 over the strongest baseline by 4.2 percentage points (pp) on Nemotron 3 Ultra and 2.0 pp on Qwen 3.6 27B; on Nemotron, it takes only 15.3% tokens to match the best baseline performance. In RL training experiments, CER exceeds full-rollout TMax by 1.9 pp while using 52.7% fewer online policy-and-judge tokens. Together, CER provides an interpretable, efficient, and dense evaluation method for long-horizon coding agents.

1 INTRODUCTION

Long-horizon coding agents receive success evidence late, increasing inference and learning costs and leaving terminal rewards too sparse to distinguish different kinds of progress. CER predicts terminal reward from visible prefix behavior using task- and stage-specific rubrics, then uses those scores for test-time scaling and verifier-free online RL.

1 INTRODUCTION

  • Long horizons delay evidence of success, raising inference and learning costs while allowing early mistaken hypotheses to shape later actions.Sparse terminal zeros can treat a trajectory that localizes a bug but fails to finish like one that browses or edits without diagnosis.Source: §1 · p. 1
  • Early terminal-reward prediction estimates an unfinished prefix’s future terminal reward from observed tool outputs and the current workspace diff, without future actions or evaluation.Unlike a process reward model, it predicts trajectory-level eventual success rather than a local action’s reward.Source: §1 · p. 1
  • CER builds a shared weighted rubric for a group of candidate prefixes, scores each against visible criterion-level evidence, and makes rubric value dependent on task and stage.Reproducing a failure or validating an edit can indicate success, while unsupported edits or repeated browsing can indicate failure.Source: §1 · p. 1
  • For test-time scaling, CER ranks visible continuations to direct limited generation toward promising candidates; for RL, it scores prefixes to form group-relative rewards without online terminal verification.The RL rubrics are constructed offline from historical rollouts.Source: §1 · p. 2
  • On SWE-bench Verified, in the TTS experiment, CER raises RM@8 over strongest baselines from 63.4% to 67.6% on Nemotron 3 Ultra and 69.4% to 71.4% on Qwen 3.6 27B.On Qwen 3.5 9B in a three-fold RL experiment, CER exceeds complete-rollout TMax (51.6% versus 49.7%) while using 52.7% fewer online tokens during RL.Source: §1 · p. 2

2 RELATED WORK

Prior work offers dense process rewards, execution-free reward models, and rubric-based evaluation, but the approaches described here have limitations for early prediction or contextual judging. CER instead builds task- and stage-conditioned criteria without a separate value model or white-box probe.

2 RELATED WORK

  • Process reward models provide dense intermediate feedback as a supplement to test-based verifiers; AgentPRM estimates step values from Monte Carlo continuations.Other examples use temporal-difference targets or internal policy representations to estimate prefix rewards.Source: §2 · pp. 2–3
  • Monte Carlo process reward models estimate state values but require sampled suffix rollouts, while learned prefix critics can be sensitive to policy and scaffold shifts.CER instead constructs task- and stage-conditioned criteria without training a separate value model or requiring white-box access.Source: §2 · p. 3
  • Execution-free methods include learned trajectory classifiers, simulated environments, fixed or task-adaptive rubrics, and early-deficiency judgments, but the described methods either require complete trajectories or underuse judging context and memory.The passage also notes test-verifier errors and a gap between verifier rewards and user intent.Source: §2 · p. 3

3 METHODOLOGY

CER scores unfinished prefixes using task-conditioned rubrics and criterion evidence, with the required prediction quality depending on downstream use. It develops experience-guided rubrics for test-time scaling and offline teacher rubrics for verifier-free online RL.

3.1 CER

  • CER defines terminal success probability from a prefix as its ideal value, then uses a judge and task-conditioned rubric set to predict early reward from the visible prefix.Rubric criteria describe observable properties such as locating relevant code, checking regressions, or recovering from an unproductive direction.Source: §3.1 · p. 3
  • For within-group selection, CER need not be calibrated across tasks or stages: preserving prefix pairwise order is sufficient when downstream use depends on relative ranking.This relaxed condition lets TTS select the same top candidates even if score gaps differ from true terminal values.Source: §3.1 · p. 3

3.2 CER FOR TEST-TIME SCALING

  • For TTS, CER retrieves related historical experience and compares continuations from a common prefix to generate rubrics, score candidates, and expand selected continuations.Experience is retained when it improves pairwise judging accuracy; regression stopping, multiple beam parents, and tie-breaking rubrics support search.Source: §3.2 · p. 4

3.3 CER FOR REINFORCEMENT LEARNING

  • For RL, a strong teacher synthesizes task-specific rubrics offline from rollout groups and verifier outcomes, refining candidates when their rubric scores improve pairwise agreement.The setup retains verifier-free rubric rewards during online training because preliminary adaptive rubrics from a weaker model provided weak supervision.Source: §3.3 · p. 5
  • CER searches task-specific rubric weights to improve agreement with verifier rewards, reaching 72.88% pairwise accuracy from 66.41% on 3,410 held-out pairs.It also uses group-level abstention and a score-range threshold to discard groups without reliable relative learning signal.Source: §3.3 · p. 5
  • Retained RL groups stop at a fixed turn budget and use centered group advantages from rubric scores, so no terminal rollout or online test verifier is needed for the policy-update reward.CER changes only the advantage supplied to the shared token-level binary-TV DPPO objective.Source: §3.3 · p. 5

4 EXPERIMENTS

Experiments use SWE-bench Verified for TTS with two models and for three-fold RL with Qwen 3.5 9B. CER is compared with reward-model and rubric baselines, while results are reported through resolved-rate metrics and token costs.

4.1 MODELS AND DATASET

  • CER matches the best execution-free baseline using 15.3% of required tokens on Nemotron 3 Ultra and 74.2% on Qwen 3.6 27B, and improves over it by 4.2 and 2.0 pp, respectively, with no more computation.The benchmark covers all 500 SWE-bench Verified instances, with a leave-one-out experience bank for TTS.Source: §4.1 · p. 5, Table 1 · p. 6

4.2 BASELINES

  • TTS baselines include learned reward models, a fixed behavioral rubric, task-adaptive rubrics, and a strengthened Adaptive Rubrics baseline that observes trajectory-group evidence but lacks an experience bank.For RL, TMax uses full rollouts and verifier rewards, TMax-40 obtains terminal rewards but optimizes only the first 40 steps, and CER stops at 40 steps using prefix rubric scores.Source: §4.2 · pp. 6–7

4.3 EVALUATION

  • TTS reports RM@k as the average resolved rate of the highest-reward rollout among k samples; RL reports mean resolved rate across sampled test rollouts with 95% sampling confidence intervals.Reported token costs count policy and judge input/output tokens, with TMax costs including policy tokens required to obtain terminal rewards.Source: §4.3 · p. 7

5 RESULTS

CER improves test-time scaling quality per token and exceeds full-rollout TMax on the reported RL test sets. Prefix behaviors provide useful learning signals, though the RL comparison uses a limited task pool and a controlled TMax baseline.

5.1 TEST-TIME SCALING

  • CER improves RM@8 over the best complete-trajectory baseline by 1.4 pp on Nemotron 3 Ultra at 20 steps and 0.8 pp on Qwen 3.6 27B at 40 steps.At 60 steps, CER reaches RM@8 of 67.6% on Nemotron with 64.8% fewer tokens, and 71.4% on Qwen; CER closes 51% and 49% of the attainable gap, versus 27% and 35% for the strengthened baseline.Source: §5.1 · pp. 7–8
  • Across both policies, increasing the prefix budget generally improves CER, while agents’ self-correction is usually local rather than a revision of an incorrect root-cause hypothesis.Agents may recover from unproductive inspection by opening more relevant files, but rarely abandon an initially incorrect root-cause hypothesis.Source: §5.1 · p. 8

5.2 REINFORCEMENT LEARNING

  • CER exceeds full-rollout TMax by 1.9 pp on test sets while using 52.7% fewer online policy-and-judge tokens during RL.At 384 steps, validation resolved rate is 57.8%, 3.5 pp above TMax; CER reaches 58.7% at 512 steps with a token budget equivalent to TMax’s 256 steps.Source: §5.2 · p. 8

5.1 TEST-TIME SCALING

  • CER’s transfer across models is asymmetric: stronger-model experience improves weaker-model TTS, while reverse transfer is detrimental.The table reports resolved RM@8 (%) at each budget k on the same 100-instance subset.Source: Table 3 · p. 8

5.2 REINFORCEMENT LEARNING

  • Prefix-only optimization is effective because policy uncertainty is higher in turns 1–40 than in turns 41–80, and prefix behaviors can distinguish groups with identical terminal rewards.Mean token entropy is 0.399 nats over turns 1–40 and 0.312 over turns 41–80; terminal-reward TMax-40 cannot learn from zero-variance groups.Source: §5.2 · p. 8
  • The RL comparison is a controlled baseline, not a reproduction of TMax’s final model: each SWE-bench fold uses 250 tasks rather than TMax’s 14,600 synthetic tasks.In this data-limited setting, TMax reaches its best mean validation rate by 128–256 steps and quickly saturates.Source: §5.2 · p. 8

6 ANALYSIS

The analysis finds that experience transfer depends on model direction, rubric rewards only moderately agree with terminal verifiers, and generated rubrics adapt their stage emphasis and criteria to individual tasks.

6.1 EXPERIENCE BANK TRANSFERABILITY ACROSS MODELS

  • Qwen-derived experience improves Nemotron TTS by 1–2 pp at every round, whereas Nemotron-derived experience reduces Qwen performance by 5–7 pp.Retrieval keywords were replaced with target-model keywords wherever source tasks aligned, separating experience-content effectiveness from retrieval accuracy.Source: §6.1 · p. 9

6.2 RUBRIC JUDGE AGREEMENT WITH VERIFIABLE REWARD

  • Across 21,938 online rubric–verifier pairs, agreement is 60.9%, with training averages ranging from 53.9% to 70.1%.The authors interpret this as a related but distinct ordering of prefix behaviors, and note that online agreement is not directly comparable to offline held-out accuracy because policy updates cause distribution drift.Source: §6.2 · p. 9

6.3 QUALITATIVE AND QUANTITATIVE ANALYSIS OF RUBRICS

  • Across 1,891 rubrics for 500 tasks, 48.2% emphasize implementation and editing, 29.0% validation, 22.5% exploration and diagnosis, and 0.3% other risks.In 89.2% of tasks, one stage receives more than half the weight, with a median dominant-stage share of 74.2%; criteria also vary by task even within the same rubric category.Source: §6.3 · p. 9

7 CONCLUSION

The paper presents CER as a way to predict terminal reward from trajectory prefixes and use those rewards in test-time search and terminal-free group-relative RL. It also cautions that automated coding and rubric judgments require human oversight before deployment.

7 CONCLUSION

  • CER constructs task-conditioned rubrics from retrieved historical experience and sampled within-group rollouts for test-time search and terminal-free group-relative RL.The experiments report improved RM@8 over best baselines on both TTS models and a 1.9 pp RL test advantage over full-rollout TMax with 52.7% fewer online policy-and-judge tokens.Source: §7 · p. 9
  • The authors report using generative AI for experimental artifacts, implementation, analysis, and writing support, while not using it to develop the conceptual framework, hypotheses, methodology, or experimental design.They state that they manually verified sources, tested AI-assisted code, audited results, and reviewed AI-assisted text and analyses.Source: AI USE STATEMENT · p. 10
  • Automated coding systems may produce incorrect or insecure patches, and rubric judgments are imperfect; the authors recommend human review before deployment.They state that rubric judgments should not replace security review or maintainer judgment.Source: ETHICS STATEMENT · p. 10

A LIMITATIONS

CER has upfront experience-bank construction costs and adds synchronous judging pauses during test-time scaling. Reuse spreads the construction investment across tasks, while asynchronous scheduling could reduce the critical-path bottleneck.

A LIMITATIONS

  • CER requires historical trajectories to be collected and terminally evaluated before its experience and supervised rubrics can be reused across tasks and stages.The benefit grows with reuse, and experience summarized from stronger models can guide weaker ones.Source: A LIMITATIONS · p. 13
  • Synchronous rubric judging changes TTS critical-path cost from O(kN + j) to O(kN + Rj), though parallel generation and pipelining can bring either pipeline near O(N).Consistently realizing that benefit requires better asynchronous scheduling infrastructure.Source: A LIMITATIONS · p. 13

A.1 OFFLINE RUBRIC & EXPERIENCE BANK CONSTRUCTION COST

  • When m = e and Cjudge ≤ Ctrain, CER’s construction cost is no greater than reward-model training under the stated accounting.The comparison depends on teacher inference and reward-model training costs; the experiments use m = 3 passes.Source: §A.1 · p. 13

B RELATED WORK (CONT’)

  • Unlike memory methods that use experience to improve generation, CER retrieves experience to define discriminative judging criteria for unfinished long-horizon coding prefixes.ARBOR adds process scores to outcome rewards for short-horizon QA trajectories scored after completion, so unfinished-prefix terminal-reward prediction does not arise there.Source: B RELATED WORK (CONT’) · p. 13

C THEORETICAL PROOFS

The proofs distinguish the score properties needed for ranking prefixes in TTS from those needed for stable RL updates, then describe robust rubric-weight optimization. Pairwise order suffices for TTS ranking, but RL also requires control of reward gaps and batch-level directional bias.

C.1 OFFSET INVARIANCE AND ORDER INVARIANCE FOR TTS

  • Mean centering removes a group-specific score offset, and preserving pairwise order preserves the Top-k candidates for rank-based TTS when the boundary values are distinct.Exchanging tied candidates at the boundary leaves the rank-based objective unchanged.Source: §C.1 · p. 14

C.2 PAIRWISE FORM OF A DPPO UPDATE

  • Pairwise order consistency preserves the sign of each pairwise gradient contribution in DPPO, but reward-gap distortion can rescale contributions and cross-pair cancellations remain uncontrolled.A sufficient batch-level stability condition therefore adds unbiased rubric-induced updates across independently sampled groups.Source: §C.2 · p. 14, §C.3 · p. 15

C.3 A BATCH-STABILITY REQUIREMENT FOR RUBRIC REWARDS

  • Batch averaging reduces non-systematic reward distortion by a factor of B, but pairwise accuracy alone cannot prevent systematic directional bias from persisting as the batch grows.The bound is stated for independently sampled groups under the additional sufficient batch-level stability condition.Source: §C.3 · p. 15

D ROBUST RUBRIC WEIGHT OPTIMIZATION

  • Rubric weight optimization keeps rubric text, anchors, polarity, and scoring rules fixed while searching allowed weights and swaps against a robust objective.Multiple initial vectors are used to reduce sensitivity to local optima.Source: D ROBUST RUBRIC WEIGHT OPTIMIZATION · pp. 15–16
  • A 66.41% to 72.88% increase in held-out pairwise accuracy follows optimizing rubric weights on 13,639 trained pairs, with 3,410 held-out pairs sharing historical trajectories but not pairs.The objective averages pairwise agreement over perturbed rubric scores to improve robustness to judging noise.Source: D ROBUST RUBRIC WEIGHT OPTIMIZATION · pp. 15–16

E.1 MODELS, CHECKPOINTS, AND DATA

The experiments use five LLMs across rollout, rubric-generation, and judging roles, with benchmark evaluation and RL folds specified in the setup. Retrieval combines task and state context with stored experience, while rubric scores also govern regression checks and RL group filtering.

E.1 MODELS, CHECKPOINTS, AND DATA

  • Five LLMs serve distinct experiment roles: Nemotron 3 Ultra and Qwen 3.6 27B handle TTS rollouts, rubric generation, and judging; Qwen 3.5 9B is the RL base and rollout policy.GPT-5.6 Sol generates TTS experience and RL rubrics, while GPT-5.6 Luna judges RL rubrics.Source: §E.1 · pp. 16–17
  • Evaluation covers all 500 SWE-bench Verified instances, using leave-one-out experience for TTS and three repository-separated RL folds, with Django shared across test and development in two folds.Baselines share eight independently sampled complete rollouts per task; CER expands beam parents in 20-step rounds and reports selected continuations at its step budget.Source: §E.1 · p. 17

E.2 FULL RL TRAINING HYPERPARAMETERS

  • The RL training and evaluation recipe follows TMax, with the full hyperparameter configuration shared across methods unless explicitly distinguished.The full configuration is reported in Table 5.Source: §E.2 · p. 17, Table 5 · p. 18

E.3 RETRIEVAL AND RUBRIC SCORE CONFIGURATION

  • Retrieval queries combine task description, stage, task contract, and state summary, fuse BM25 rankings over experience content and keywords, and retrieve six experiences.A continuation is regressed when its parent-comparison score falls below 0.5; RL groups with less than a 0.18 rubric-score range are filtered.Source: §E.3 · pp. 17–18

F.1 MORE METRICS OF TTS EXPERIMENTS

On fully unsubmitted trajectory subsets, CER outperforms the strongest terminal-reward baselines at almost all trajectory lengths. The section also notes limits on experience-transfer scope and on interpreting TTS metrics.

F.1 MORE METRICS OF TTS EXPERIMENTS

  • CER shows superior TTS performance at almost all trajectory lengths on subsets where every candidate remains unsubmitted.Table 6 compares the same task subset for CER and the two strongest terminal-reward baselines at each cutoff, reporting only subsets larger than 50 tasks.Source: §F.1 · p. 18, Table 6 · p. 19
  • 50.7–60.4% of retrieved experiences come from the target task’s repository, leaving fully cross-repository transfer an open direction.Longer trajectories require more in-domain experience because descendants of the same parent prefix have less variance and need more discriminative, related experience.Source: §F.1 · pp. 18–19
  • Accpair is not directly comparable across CER and baselines, and pointwise accuracy is unreliable because prefix value is latent and one suffix yields only one Bernoulli sample.The authors therefore use RM@k for TTS; reporting 95% confidence intervals for rollout-generation variance would require prohibitively costly repeated sets of eight candidates per task.Source: §F.1 · p. 19

F.2 TTS COMPONENTS ABLATION

Ablations identify sibling comparison as the most important CER component and diversified samples as the most important search component. Regression stopping helps Nemotron but not Qwen in this experiment.

F.2 TTS COMPONENTS ABLATION

  • Removing sibling comparison lowers average RM@8 by 3.0 pp on Qwen 3.6 27B and 4.0 pp on Nemotron 3 Ultra.The ablation keeps candidate prefixes and retrieved experiences but scores each continuation separately rather than jointly judging siblings; cross-continuation contrast is most useful when early candidates explore diverse hypotheses.Source: §F.2 · p. 19
  • Removing retrieved experience lowers average RM@8 by 2.3 pp on Qwen 3.6 27B and 1.3 pp on Nemotron 3 Ultra.The authors note that its benefit varies with how effectively the judge can use the experience.Source: §F.2 · p. 19
  • CER retrieves slightly more experience from the target task’s repository than from other repositories.The ablation uses the same sampled 100-instance SWE-bench Verified subset as Section 6.1.Source: Table 7 · p. 19, §F.2 · p. 19
  • Removing beam parents lowers average RM@8 by 2.3 pp on both models; removing tie-breaking lowers it by 0.5 pp on Nemotron and 0.8 pp on Qwen.These results identify diversified samples as the most important CER-guided search component.Source: §F.2 · p. 19
  • Removing regression stopping lowers average RM@8 by 1.5 pp on Nemotron 3 Ultra but raises it by 1.3 pp on Qwen 3.6 27B.Most Qwen trajectories submit within two or three rounds, so pruned branches are rarely recovered; regression stopping is most useful for longer searches such as Nemotron’s.Source: §F.2 · pp. 19–20

F.3 COMPARISON BETWEEN MEMORY-ENHANCED EVALUATION AND GENERATION

On a 100-task subset, policy skill memory performs slightly below the no-skill-bank average, while CER@20 performs better. CER’s evaluation memory can leverage test-time search, whereas policy memory assists a single generation.

F.3 COMPARISON BETWEEN MEMORY-ENHANCED EVALUATION AND GENERATION

  • CER@20 resolves 63% of Nemotron 3 Ultra instances and 73% of Qwen 3.6 27B instances, versus 58% and 68% with policy skill memory.Without the skill bank, average performance on the same subset is 58.5% and 68.1%, respectively.Source: §F.3 · p. 20
  • CER@20 uses 26% more tokens than a single rollout on Nemotron 3 Ultra: 3.96M versus 3.15M tokens per instance.The authors contrast evaluation memory, which can leverage test-time search, with a policy bank that can help only a single generation.Source: §F.3 · p. 20
  • Policy skill memory is strongly task-dependent, while evaluation memory uses recurring behavioral evidence informative across repositories and tasks.The paper acknowledges CER’s out-of-domain generalization limitation but reports that it is more generalizable than policy-focused memory.Source: §F.3 · p. 20

F.4 TRAINING DYNAMICS

This section examines CER’s training signal and the evidence needed for useful early prediction, while defining how prompts generate and repair task-specific rubrics, score continuation evidence, and abstain from prefix comparisons. It emphasizes fixed-cutoff, visible evidence and guards against using private outcomes or future actions.

F.4 TRAINING DYNAMICS

  • CER’s entropy loss stayed stable while verifiable-reward entropy loss rose and became unstable; CER rollouts also shortened as TMax and TMax-40 rollouts lengthened.TMax and TMax-40 filtered substantially more groups than CER. Mean reward stayed flat across methods because dynamic task sampling changed batch composition, so validation curves—not mean sampled reward—tracked learning progress.Source: §F.4 · p. 20, Figure 5 · p. 21
  • 43.8% of CER’s sampled groups had identical binary resolution across eight trajectories, yet CER trained on 83.3% of those groups because rubric scores differed.These groups represented 40.5% of all groups CER trained on, with a mean weighted rubric range of 0.43 against the 0.18 abstention threshold.Source: §F.4 · p. 20
  • CER’s shorter rollouts were not accompanied by increased early submission: it submitted within 40 turns less often than TMax (27.1% versus 32.7%), with the rate falling from 24.8% to 23.9% across checkpoint halves.Program-free success claims were rare and similar across methods (0.4–0.6%).Source: §F.4 · p. 21

F.5 CAN CER BE USED AS PROCESS REWARD MODEL?

  • CER’s normalized rubric score rises from 0.431 at step 10 to 0.852 at step 100 while binary cross-entropy worsens from 0.755 to 3.514, making it unsuitable as a general process reward model.The scores become more confident without becoming more predictive of terminal success; in its current form, CER is suited to comparing candidates at a fixed prefix length.Source: §F.5 · p. 21

F.6 HOW EARLY CAN WE PREDICT?

  • CER policies trained for 10, 20, 30, and 40 steps resolved 48.1%, 48.3%, 49.8%, and 52.7% of the test set, respectively.Median entry into editing and post-edit validation occurred at steps 22 and 28, respectively, so 30- and 40-step prefixes could include both implementation and validation evidence.Source: §F.6 · p. 22
  • The fraction of reference-patch target files touched reached only 0.608 AUROC at step 25, showing exploration alone was a weak predictor of terminal resolution.Failed trajectories could inspect the right files without implementing a correct solution, so useful prediction required prefixes with editing and validation evidence.Source: §F.6 · p. 22

G PROMPT TEMPLATES

  • CER’s rubric-generation prompt asks for one discriminative, nonredundant criterion grounded in observable differences between current continuations, with concrete scales and positive or negative polarity.Existing rubrics and retrieved experiences may be reused or adapted only when relevant to the current continuations; the instructions caution against treating retrieved experiences as privileged outcome evidence.Source: G PROMPT TEMPLATES · p. 26
  • Continuation judges score only the specified criterion using exact visible trajectory evidence, returning evidence and score rather than inferring unstated facts or judging general quality.The score uses the rubric’s required scale, including inverted scales for negative rubrics.Source: G PROMPT TEMPLATES · p. 29
  • Experience cards target one diagnosed rubric or judge failure and specify visible activation evidence, contradictory evidence, exclusions, and an abstention boundary without exposing privileged outcomes.The refinement prompt requires a substantive correction when the previous card failed to improve both target metrics.Source: G PROMPT TEMPLATES · pp. 30–31
  • A conservative gate abstains only with strong visible evidence that all eight prefixes are materially equivalent; any credible task-relevant difference blocks abstention.Workflow stage or verbosity alone does not establish reward similarity, while a terminal non-patch submission cannot be rescued by valid workspace contents.Source: G PROMPT TEMPLATES · p. 31
  • Initial rubric portfolios contain 1–6 complementary criteria grounded in cutoff-visible evidence, with legitimate ties preserved and private outcomes used only as supervision.Rubrics should cover independently observable dimensions without redundancy or proxies such as group IDs, reward ordering, or stage position.Source: G PROMPT TEMPLATES · p. 32
  • Failed rubric candidates are repaired using judge outcomes and visible trajectories while preserving their intended semantic dimension; deletion and abstention are reserved for downstream weight optimization.Repairs can adjust evidence authority, hard gates, contradiction caps, anchors, polarity, or scope, without encoding private values or future outcomes.Source: G PROMPT TEMPLATES · pp. 32–33

Related papers

Loading 2609.31995v1…