PaperlayerView exact revision on arXiv

Source-linked AI summary

SWE-Critix: Post-Training Generalist Verifiers for Assessing Coding-Agent Trajectories

Jiajun Hu, Chun Yong Chong

arXiv:2610.13356v1cs.AI

TL;DR

Coding-agent trajectories can be long and lack readily available success oracles, while training-based, process-oriented generalist verification remains underexplored. SWE-CRITIX combines supervised fine-tuning and reinforcement learning to train a verifier that judges completion and explains its reasoning; accuracy reaches 78% after RL, up from 57% for the base model.

  • Problem

    Reliable verification is needed when task success lacks a readily available oracle and trajectories contain hundreds of interactions; training-based, process-oriented generalist verification remains underexplored.

    Source: §I · p. 1

  • Method

    SWE-CRITIX combines SFT and RL, using teacher-generated comparisons of successful and failed trajectories to supervise reasoning before training on samples rewarded using unit-testing results.

    Source: §I · p. 1, B. Chain-of-Thought (CoT) Annotation · p. 3

  • Results

    Accuracy increases from 57% for the base model to 78% after reinforcement learning on the held-out test set.

    Source: §I · p. 2

  • Takeaways & Limitations

    SWE-CRITIX predicts task completion and provides a rationale explaining its judgment, and the authors release the work on GitHub and Hugging Face.

    Source: §I · pp. 1–2

    The current evaluation focuses on discriminative ability; the authors plan more comprehensive evaluations across five dimensions.

    Source: §IV · p. 4

Abstract

from arXiv · show

Coding agents are increasingly deployed for end-to-end software engineering tasks, where a readily available oracle evaluator that defines the success criteria is often absent. This motivates verifications for coding agents that can reliably judge and explain their behavior. However, verification is non-trivial as coding agents are increasingly capable of solving long-horizon software engineering tasks. Their execution trajectories can be extremely long, containing hundreds of iterative interactions with code repositories and external tools. Existing verifiers either rely on general-purpose LLMs without specialized training or train dedicated critics that predominantly focus on producing scalar rewards, highlighting the relatively limited exploration of training-based generalist verification. We introduce SWE-Critix, a post-training pipeline for training generalist verifiers of coding-agent trajectories. Given a task and its execution trajectory, the learned verifier predicts the completion label and generates a natural-language analysis that explains its judgment. Our pipeline combines supervised fine-tuning (SFT) with reinforcement learning (RL) and trains a series of 30B models on trajectories from CodeForge-Preview. On a held-out test set, SWE-Critix improves classification accuracy from 57% for the base Qwen3-30B-A3B reasoning model to 66% after SFT and 78% after RL. We release our models, datasets, and training code to facilitate further research.

I. INTRODUCTION

Coding-agent trajectories are difficult to verify when success criteria lack a readily available oracle and executions span many interactions. SWE-CRITIX trains a generalist verifier to judge completion and explain its rationale, improving held-out accuracy after post-training.

I. INTRODUCTION

  • Coding-agent verification is needed when success criteria lack a readily available oracle and trajectories interleave user inputs, feedback, and model responses.LoLBench trajectories span 53–532 turns and contain millions of tokens on average.Source: §I · p. 1
  • Training-based, process-oriented generalist verification remains relatively underexplored, motivating SWE-CRITIX’s post-training verifier.Given a task and execution trajectory, it predicts successful completion and explains the rationale underlying its judgment.Source: §I · pp. 1–2
  • Within this category, SWE-CRITIX uses 119K training samples and 30B models, compared with two existing approaches’ 2–8K samples and 4–8B models, respectively.Source: §I · p. 2
  • Accuracy rises from 57% for the base model to 78% after reinforcement learning on the held-out test set.The comparison is reported for the Qwen3-30B MoE verifier.Source: §I · p. 2

II. SWE-CRITIX

SWE-CRITIX trains a verifier to assess issue-solving trajectories, producing a rationale before its binary completion judgment. Its pipeline uses teacher-generated comparisons for SFT and unit-testing results as RL reward signals.

II. SWE-CRITIX

  • The verifier takes an issue-solving trajectory, produces a chain-of-thought rationale, and then judges whether the issue was resolved.The rationale is described as informing the binary decision.Source: §II · p. 2
  • A teacher compares successful and failed trajectories for the same issue, summarizing procedures, mistake patterns, and factors associated with success or failure.These summaries provide reasoning supervision in a small set of SFT samples.Source: §II · p. 2
  • RL uses many samples without CoT annotations and unit-testing results as ground truth for the reward signal.Source: §II · p. 2

A. Data Collection

The dataset comprises test-verified coding-agent trajectories, filtered and divided by issue into test, SFT, and RL sets. The split design reserves issues for testing and balances RL classes while maximizing issue coverage.

A. Data Collection

  • CoderForge-Preview contains 258K trajectories across 51K issues and 1,655 repositories, with 155K passes and 103K failures.Trajectories were generated using OpenHands with Qwen3-Coder-480B across three downstream task sources with executable gym environments.Source: A. Data Collection · p. 2
  • After discarding trajectories over 64K tokens or lacking a submitted patch, the authors filter out 16% of the data.Source: A. Data Collection · p. 2
  • The test set uses all trajectories from a random 20% of issues; SFT selects one successful and one failed trajectory per eligible remaining issue.RL uses remaining failed trajectories and samples successful ones round-robin across issues until classes are balanced, maximizing distinct-issue coverage.Source: A. Data Collection · pp. 2–3

B. Chain-of-Thought (CoT) Annotation

SWE-CRITIX uses a teacher LLM to create structured reasoning traces by comparing successful and failed trajectories, then adapts those traces for pointwise evaluation. CoT annotation is limited to the SFT set.

B. Chain-of-Thought (CoT) Annotation

C. Supervised Fine-Tuning (SFT)

SWE-Critix uses teacher-generated reasoning traces to supervise its verifier on successful and failed trajectories, then fine-tunes the model on those examples. The SFT stage minimizes response cross-entropy and produces the checkpoint used for later training.

C. Supervised Fine-Tuning (SFT)

D. Reinforcement Learning (RL)

RL trains the verifier on examples without CoT annotations, using unit-test outcomes to reward correct completion judgments. The method begins from the SFT checkpoint and uses GRPO to optimize the policy.

D. Reinforcement Learning (RL)

  • RL uses the remaining 91.3% of training data, which contains unit-test results but no CoT annotations.The SFT stage uses a small fraction of the training data with CoT annotations to teach reasoning patterns first.Source: D. Reinforcement Learning (RL) · p. 3
  • The rule-based outcome reward is +1 when the predicted judgment agrees with unit-test results and −1 otherwise.The reward evaluates the extracted final answer against whether the trajectory passes tests.Source: D. Reinforcement Learning (RL) · pp. 3–4
  • RL starts from πSFT and uses GRPO with group size 8 to optimize the policy objective.One epoch comprises 3,396 iterations and takes approximately 533 hours.Source: D. Reinforcement Learning (RL) · pp. 3–4

III. PRELIMINARY EVALUATION

On the held-out test set, RL improves trajectory-verification accuracy over SFT, while failed trajectories remain harder to recognize. The evaluation also finds weaker performance on R2E-Gym than on the other two subsets.

III. PRELIMINARY EVALUATION

  • Each model is evaluated once on more than 43K long-trajectory test samples, with accuracy and class-specific precision, recall, and F1 reported.Source: §III · p. 4
  • 78% overall test accuracy after RL improves on 66% after SFT.The results are reported for the test set and its three subsets.Source: §III · p. 4
  • RL raises positive recall from 67% to 85% without substantially sacrificing negative recall, but negative recall remains 64% overall.The authors attribute the remaining difficulty to diverse failure patterns and subtle failure signals.Source: §III · p. 4
  • Performance on R2E-Gym is notably lower than on SWE-Rebench and SWE-Smith, possibly because R2E-Gym has less data.Source: §III · p. 4

IV. FUTURE WORK

The authors outline five directions for broadening SWE-CRITIX’s training and evaluation. These include varying training data and methods, exploring downstream uses, expanding baselines, and analyzing reasoning quality and errors.

Training Data

  • Training data is limited to CodeForge-Preview samples generated with OpenHands and Qwen3-Coder-480B; future work will test other sources, model and harness combinations, and class ratios.Proposed ratios include 2:1, 1:2, and 4:1.Source: §IV · p. 4

Training Methods

  • Whether RL alone can train an effective verifier remains open; planned studies include cold-start RL and rewards incorporating intermediate reasoning signals.The cold-start baseline removes CoT annotation and SFT from the pipeline.Source: §IV · p. 4

Downstream Applications

  • Planned downstream studies will test whether the verifier can help correct failed trajectories or serve as a proxy reward when ground-truth tests are unavailable.Source: §IV · p. 4

Baselines

  • The authors plan comparisons with frontier open- and closed-source models and representative verification methods.Source: §IV · p. 5

Qualitative Analysis

  • Future qualitative analysis will assess whether reasoning traces are coherent and evidence-grounded, and examine incorrect judgments to understand their error sources.Source: §IV · p. 5

V. RELATED WORK

Prior coding-agent verification work spans training-free evaluators and specialized trained critics, and differs in both what is evaluated and how verifiers are built.

V. RELATED WORK

  • Training-free verification uses general-purpose LLMs to assess executions through test generation [7], [9], rubric grading [10]–[12], verification scaling [8], metamorphic relations [16], failure attribution [13], or anti-pattern detection [14], [15], [17], [18].Source: §V · p. 5
  • Training-based verification trains specialized critics using paradigms including TD learning, [26], next-token prediction [10], [19]–[23], [28]–[31], multi-label classification [27], DPO [30], Bradley-Terry [24], and policy gradient [28], [31].Source: §V · p. 5
  • The literature’s two categories distinguish whether a specialized model is trained, while verification approaches also vary in their targets and learning paradigms.Source: §V · p. 5

VI. CONCLUSION

SWE-CRITIX combines SFT and RL to train a process-level generalist verifier for coding-agent trajectories. Preliminary CodeForge-Preview experiments report improved discriminative ability.

VI. CONCLUSION

  • SWE-CRITIX integrates supervised fine-tuning and reinforcement learning in a post-training pipeline for a process-level generalist verifier.Source: §VI · p. 5
  • The verifier is designed to assess coding-agent trajectories at the process level.Source: §VI · p. 5
  • Preliminary experiments on CodeForge-Preview trajectories demonstrate substantial improvement in the model’s ability to distinguish successful from failed trajectories.Source: §VI · p. 5

Related papers

Loading 2610.13356v1…