PaperlayerView exact revision on arXiv

Source-linked AI summary

Quality, Not Just Outcome: A Structural Scoring Framework for Selecting Coding-Agent Trajectories for Supervised Fine-Tuning

Sida Li, Yuanxin Liu, Shicheng Li, Linli Yao, Xu Sun, Haoli Bai, Xiaohui Li

arXiv:2610.13152v1cs.AI

TL;DR

Outcome-only filtering does not distinguish process quality among successful coding-agent trajectories, even though supervised fine-tuning imitates their full interactions. The paper introduces TQS, a structural event-log scorer, and reports that top-ranked trajectories outperform random or bottom-ranked selections. The authors conclude that trajectory structure is a scalable supervision signal beyond task outcome alone.

  • Problem

    Outcome-level filtering does not distinguish process quality among successful trajectories, although fine-tuning exposes students to the full interaction process and existing selection approaches do not characterize search organization over time.

    Source: §1 · p. 1

  • Method

    TQS scores trajectory structure through seven linear-time measures of information focus, intent–action alignment, and policy dynamics, without relying on LLM judges or external reward models.

    Source: §1 · pp. 1–2

  • Results

    TQS-Top reaches 27.6 pass@1, 2.8 points above Random-K and 3.2 points above the base model on SWE-Bench Verified.

    Source: §5.1 · p. 7

  • Takeaways & Limitations

    Trajectory structure offers a useful, scalable supervision signal beyond task outcome, and low-TQS trajectories can hurt downstream performance.

    Source: §6 · p. 9

    Reported SFT comparisons focus on a 30B model because preliminary 7B experiments showed no stable selector differences; larger backbones remain untested.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Open coding agents are commonly trained through supervised fine-tuning on interaction trajectories whose final patches pass the benchmark test suite. However, this outcome-level filtering criterion ignores process quality: trajectories that eventually succeed may still contain inefficient interaction patterns such as repetitive file inspection, unstable tool usage, or delayed modification behavior, all of which become part of the imitation target. We propose TQS (Trajectory Quality Score), a structural scoring framework for coding-agent trajectories that operates directly on trajectory structure, requires no LLM judge, and runs in linear time. TQS characterizes trajectory quality through structural signals organized around structured convergence: information focus, intent--action alignment, and policy dynamics. Using 155,144 successful trajectories from a public open-license corpus, we rank trajectories by TQS and fine-tune coding agents on filtered subsets. Models trained on top-ranked trajectories consistently outperform those trained on random or bottom-ranked subsets on SWE-Bench-Verified. Our results suggest that process-level trajectory quality is an important factor in coding-agent supervision and that structural trajectory filtering provides an effective and scalable alternative to LLM-based data selection.

1 Introduction

Outcome-based filtering can retain successful but inefficient trajectories, even though supervised fine-tuning imitates their full interaction process. TQS scores structural convergence from event logs, and top-ranked trajectories outperform random or bottom-ranked selections on SWE-Bench Verified.

1 Introduction

  • Because fine-tuning imitates reasoning traces, tool use, and patches, inefficient or unstable interactions become part of the training signal.Successful trajectories can still repeat file inspections, interrupt stalled commands, or delay meaningful edits.Source: §1 · p. 1
  • Perplexity-style scores are confounded by length and tokenization, while LLM judging is costly and does not characterize search organization over time.The paper frames trajectory selection as distinct from selecting short, single-turn samples because trajectories encode ordered exploration, editing, testing, and recovery.Source: §1 · p. 1
  • TQS scores successful trajectories using information focus, intent–action alignment, and policy dynamics as signals of structured convergence.The framework operationalizes these dimensions with seven linear-time measures; Figure 1 also shows a Dirty Pattern Indicator guardrail and a weighted score in [0, 1].Source: §1 · p. 2, Figure 1 · p. 2
  • Top-ranked trajectories consistently outperform random- or bottom-ranked selections on SWE-Bench Verified.The evaluation uses 155,144 successful trajectories and fine-tunes the same model on matched subsets with identical hyperparameters.Source: §1 · p. 2
  • TQS is presented as a scalable scoring and filtering framework that avoids LLM judges and external reward models.The stated contributions also include an open pipeline for scoring, filtering, fine-tuning, and evaluation.Source: §1 · p. 2

2 Related Work

Related work spans tool-augmented coding-agent infrastructure, SFT data selection, and process-level evaluation. TQS uses event-log temporal structure to characterize how an agent organizes its search over a trajectory.

2 Related Work

  • Coding-agent frameworks and teacher-generated datasets enabled large-scale collection of executable interaction trajectories.The cited infrastructure includes SWE-agent and OpenHands, while datasets include SWE-Gym, Lingma SWE-GPT, and CoderForge-Preview.Source: §2 · pp. 2–3
  • SFT selectors use LLM judges or training-derived signals, but were developed for short samples and require adaptation to long interaction histories.TQS instead uses temporal structure already present in the event log.Source: §2 · p. 3
  • Process-level evaluation studies intermediate reasoning, agent trajectories, policy behavior, and trajectory curation using varied approaches.Examples include faithfulness analysis, LLM-rubric selection for browser agents, entropy or behavioral-diversity methods, and software-agent curation using privileged patch information.Source: §2 · p. 3

3 Trajectory Quality Score

TQS treats trajectory quality as structured convergence, operationalized through event-log measures. Its composite discriminates paired outcomes and aligns with an external resolved-rate check.

3.1 Preliminaries and Notation

  • A trajectory is a sequence of steps represented by action type, preceding thought, tool observation, touched files, and an error-pattern flag.This tuple can be recovered from OpenHands-style event logs without model inference.Source: §3.1 · p. 3

3.2 Design Intuition and Score Construction

  • Structured convergence means search becomes organized around the emerging fix across relevant evidence, aligned thought and action, and productive exploration, execution, and recovery.Candidate components are computed in O(n) CPU time from event logs without auxiliary LLM judging.Source: §3.2 · p. 3

3.3 Components and Aggregation

  • Information-focus measures track observation concentration and narrowing file scope, while IAC scores coarse consistency between preceding thoughts and subsequent actions.OEC is interpreted alongside TTE and DPI because an unproductive loop can also concentrate observations; IAC does not assess semantic correctness.Source: §3.3 · p. 4
  • Policy-dynamics components include tool-transition entropy, the timing of the first substantive edit, and post-error changes in action type or file scope.TTE penalizes long single-tool loops; SCP gives its highest score to edits at 20–50% of relative trajectory time, and PED scores repeated action on unchanged files as zero.Source: §3.3 · p. 4
  • TQS is a renormalized weighted mean over defined trajectory components, using fixed hand-set weights (OEC 0.30, IAC 0.25, PSN/TTE/PED/DPI 0.10 each, SCP 0.05); trajectories with n < 3 or coverage below 4/7 are excluded.DPI guards against categorical failures including early stopping, no substantive edit, repeated shell-command loops, and repeated errors.Source: §3.3 · pp. 4–5

3.4 Discriminant Validity and External Sanity Checks

  • 0.732 concordance is reported for composite TQS, above every individual component; admission also rejects measures failing direction, length, or diversity checks.BUS, TGCR, and DCS-file fail concordance; DCS-Jaccard is length-correlated, and HCR is excluded for behavioral mode collapse.Source: §3.4 · p. 5, Table 1 · p. 5
  • 54.1% in D1 to 86.9% in D10 is the monotonic resolved-rate range across TQS deciles; fixed-step-band composite pair-AUC is 1.000.The fixed-step-band pair-AUC also exceeds 0.96 for OEC, IAC, and PED.Source: Figure 2 · p. 5, §3.4 · pp. 5–6

4 Experimental Setup

The experiments score successful CoderForge-Preview trajectories and construct matched training subsets. They hold the model, training recipe, and evaluation scaffold constant while varying selected trajectories.

4 Experimental Setup

  • Both runs on the same pygments LassoLexer task pass, but the high-TQS trajectory reaches the fix in 36 steps versus 73 for the low-TQS run.The low-TQS run includes many interrupts and forced process terminations; the largest component gaps are in observation focus, commitment timing, and post-error adaptation.Source: §4 · p. 6
  • 155,144 passing trajectories from CoderForge-Preview form the candidate pool; filtering leaves 38,382 eligible trajectories for matched 1,024-example subsets.Top-K, Bottom-K, and Random-K subsets control task coverage, repository coverage, and source proportions.Source: §4 · p. 6
  • SFT uses Qwen3-Coder-30B-A3B-Instruct with LoRA, and pass@1 is evaluated on SWE-Bench Verified using OpenHands v0.52.1.The training recipe is fixed across conditions, as are the evaluation scaffold and version used to generate training trajectories.Source: §4 · p. 6
  • Baselines include the untuned model, Random-K, Bottom-K, Shortest-K, DPI-only-K, GPT-4o LLM-Judge-K, and length-matched Top-LM-K.All baselines use the same eligible pool and SFT recipe.Source: §4 · p. 7

5 Results

TQS-Top improves SWE-Bench Verified performance over random and bottom-ranked supervision, and its structural ordering agrees substantially with human and model judgments. Ablations and rollout measurements connect selection quality to multiple trajectory dimensions and observable policy behavior.

5.1 Main Results

  • Length matching retains most of the Top–Random gain but does not fully explain it; the gap also grows from 1.6 points on Easy to 4.5 on Hard.Top-LM-K reaches 27.0, while Shortest-K reaches 24.2 and DPI-only-K remains 2.4 points below TQS-Top.Source: §5.1 · p. 7
  • TQS-Top matches LLM-Judge-K on Hard tasks at 6.7, while its measured corpus-scale scoring cost is far below the estimated cost of GPT-4o judging.GPT-4o judging is estimated at ∼$2,750 for 38,382 eligible trajectories; a four-worker TQS pass is estimated at about 49 minutes.Source: §5.1 · p. 7
  • TQS-Top reaches 27.6 pass@1, versus 24.8 for Random-K and 24.4 for the base model; Bottom-K falls to 21.8.TQS-Top improves 2.8 points over Random-K and 3.2 over the base model.Source: §5.1 · p. 7, Figure 4 · p. 8

5.2 Direct Validation among Successful Runs

  • Human and Sonnet-5 judgments agree with the TQS ordering on 78.7% and 75.4% of order-consistent comparisons, respectively.For complete high>mid>low rankings, agreement with TQS is 45.7% and 41.3%, above the 16.7% chance rate.Source: §5.2 · p. 8

5.3 Ablation of TQS Dimensions

  • Removing any TQS dimension reduces pass@1; removing intent–action alignment causes the largest drop, from 27.6 to 23.0.Removing information focus yields 24.6 and removing policy dynamics yields 25.8.Source: §5.3 · p. 8

5.4 Behavioral Diagnosis on Rollouts

  • The Top model averages 50.66 rollout steps versus 57.64 for Random and 63.46 for Bottom, with higher OEC and DPI than both.Rollout component patterns are not uniform: Random has slightly higher SCP and PED than Top.Source: §5.4 · p. 9, Table 6 · p. 9

6 Conclusion

TQS scores successful coding-agent trajectories using event-log structure rather than treating a passing patch as a sufficient quality signal. Top-ranked trajectories provide more effective supervision than simpler selectors, while low-ranked trajectories can hurt downstream performance.

6 Conclusion

  • TQS evaluates successful trajectories through information focus, intent–action alignment, policy dynamics, and a guardrail for categorical failure patterns.It is described as a lightweight event-log scorer.Source: §6 · p. 9
  • Top-TQS trajectories provide more effective supervision than simpler outcome-, length-, or pattern-based selection, while low-TQS trajectories can hurt downstream performance.TQS approaches rubric-based GPT-4o judging effectiveness at substantially lower selection cost.Source: §6 · p. 9
  • Successful trajectories are not equally valuable as supervision, and trajectory structure offers a scalable signal beyond task outcome alone.The paper presents repeated rescoring and corpus-scale data selection as practical at TQS's lower selection cost.Source: §6 · p. 9

Limitations

The experiments and scoring framework have scope boundaries around their data, training setup, and evaluated model scale. The worked-example table reports component scores for two trajectories using fixed weights.

Limitations

  • Experiments use CoderForge-Preview trajectories from one teacher under OpenHands; parsers, thresholds, and hand-set weights may need recalibration in other settings.The paper specifically notes different action spaces, multilingual traces, step granularities, and GUI or multimodal agents as settings requiring adaptation.Source: Limitations · p. 9
  • The fixed-budget SFT experiments select 1,024 trajectories per condition and exclude trajectories longer than 32K tokens, potentially omitting useful recovery behavior.Each condition is trained once, so pass@1 differences lack multi-seed variance estimates.Source: Limitations · p. 9
  • Downstream selector comparisons focus on the 30B model; preliminary 7B experiments showed no stable selector differences, and larger backbones remain untested..Source: Limitations · p. 9

A Worked-Example Component Scores

B Direct Validation and Semantic Gaps

Human and model rubrics substantially agree with TQS rankings, while example disagreements show semantic and verification information that structural scoring does not encode. TQS induces an ordering in score space, not a measure of trajectory similarity, and selection stability depends on the score margin.

B Direct Validation and Semantic Gaps

  • 631/861 human pairs and 656/861 Claude Sonnet-5 pairs agree with TQS; the full high>mid>low order matches 137/300 and 124/300 tasks, respectively, versus 1/6 by chance.The study sampled 300 tasks with at least three resolved runs and compared high, middle, and low TQS tiers.Source: B Direct Validation and Semantic Gaps · pp. 11–12
  • Semantic judging can favor a lower-TQS run that fixes the original bug, while a higher-TQS run addresses a secondary failure or adds checks around the wrong issue.The examples include an indexing error and a timing-dependent crash; another disagreement concerns verification, where the higher-TQS run stops after static inspection without confirming a passing test.Source: B Direct Validation and Semantic Gaps · p. 12
  • TQS is a scalar that induces a total preorder: distinct trajectories with equal scores are equivalent for selection, and nearby scores do not imply structural similarity.The score-space separation becomes a metric only after trajectories with equal scores are identified.Source: B Direct Validation and Semantic Gaps · p. 12
  • A Top-K subset is unchanged under a bounded score perturbation when its boundary margin exceeds twice the perturbation bound.Only trajectories near the selection boundary can enter or leave the subset; the paper also describes this as a conservative sufficient condition for preserving Top-K.Source: B Direct Validation and Semantic Gaps · p. 12

C Implementation Details

The implementation parses OpenHands-format trajectories, scores them with a linear-time CPU pipeline, and masks user messages and tool responses from the SFT loss. The subset sampler ranks trajectories while controlling task, repository, and source representation.

C Implementation Details

  • The parser reads OpenHands-format parquet fields, recovers tool calls and observations from messages, and normalizes file paths before scope-based measures are computed.The listed fields include trajectory and instance identifiers, repository, messages, tools, reward, and finish reason.Source: C Implementation Details · p. 12
  • Table 8 reports agreement with the TQS ordering among order-consistent pairs, while Table 9 reports concordance between pointwise rubric scores and TQS tier ordering after equal-score ties are excluded.Both tables report 95% Wilson confidence intervals and use two-sided exact binomial tests against 0.5.Source: Table 8 · p. 13, Table 9 · p. 13
  • Scoring takes 43 ms per trajectory on one worker and is estimated to require about 3.1 CPU-hours for 258,134 unique trajectories; four workers process 87.5 trajectories per second.The benchmark used 3,462 mixed-source trajectories with a median length of 53 steps; regular-expression error detection accounted for 58% of single-worker runtime.Source: C Implementation Details · p. 13
  • SFT loss is computed on assistant responses, including tool calls, while user messages and tool responses are masked; tool outputs therefore serve as conditioning context.Under the Qwen3-Coder agent template, tool calls are rendered in XML.Source: C Implementation Details · p. 13
  • The error flag uses conservative string and regex patterns to detect repeated failure patterns and post-error behavior, not semantic correctness of the final patch.Patterns include execution failures, missing-file messages, traceback markers, failed test invocations, and tool-level error statuses.Source: C Implementation Details · p. 13

C.1 Subset Sampler

  • The 1,024-trajectory subsets rank eligible trajectories by TQS while controlling task identity, repository coverage, and source proportions; the Top-1,024 and Bottom-1,024 cutoffs are 0.7061 and 0.4696.The selected regions have a score gap of 0.2365; ties are broken by trajectory_id.Source: §C.1 · pp. 13–14, Figure 5 · p. 14

D Score and Component Distributions

Eligible trajectories span a broad TQS range, and the retained components show heterogeneous marginal distributions. Their pairwise correlations are modest, while missing component values are handled through renormalized aggregation.

D Score and Component Distributions

  • The Top-1,024 cutoff is 0.7061 and the Bottom-1,024 cutoff is 0.4696, leaving a score gap of 0.2365 across the selected regions.The 38,382 eligible trajectories span a broad score range rather than collapsing into a narrow band; separation alone does not establish downstream quality.Source: D Score and Component Distributions · p. 14
  • PED has the largest component dispersion (σ = 0.232), followed by PSN (σ = 0.153) and OEC (σ = 0.118), while TTE is more concentrated (σ = 0.038).IAC varies around a mean of 0.570; SCP and DPI have means of 0.978 and 0.941, respectively, and primarily identify smaller tails of weak commitment or dirty interactions.Source: D Score and Component Distributions · p. 14, Figure 6 · p. 15
  • PED is undefined for 0.7% of eligible trajectories because its error-recovery signal requires an applicable error transition, and renormalized aggregation handles this missingness.The other six components are defined for every eligible trajectory.Source: D Score and Component Distributions · p. 14
  • The largest off-diagonal Spearman correlation is 0.320 between TTE and PED, indicating no severe marginal redundancy among retained components.The correlation analysis alone does not establish that every component is necessary; downstream dimension ablations provide a complementary test.Source: D Score and Component Distributions · p. 14, Figure 7 · p. 15

E Failed Candidate Measures

Several candidate measures were excluded for weak concordance, length confounding, or insufficient behavioral diversity. The appendix also reports the LLM-judge baseline, a length-controlled check, and the fixed-budget token-length boundary.

E Failed Candidate Measures

  • HCR passed the numerical thresholds (concordance 0.527; |ρ| = 0.014) but was excluded because 99.9% of the Top-1K subset shared one binary behavior pattern.The measure favors an explore-then-commit pattern, but its concentration failed the behavioral-diversity criterion.Source: E Failed Candidate Measures · p. 15
  • BUS and TGCR were excluded for concordance scores of 0.202 and 0.479, respectively.BUS counts useful backtracking, while TGCR measures file-navigation cycles; TGCR's result is described as close to chance.Source: E Failed Candidate Measures · pp. 15–16
  • DCS-Jaccard was rejected for strong length confounding (|ρ| = 0.729), and DCS-file was excluded for concordance of 0.238.DCS-Jaccard combines intent–action alignment with whether an observation informs the next intent; the file variant evaluates file-level alignment.Source: E Failed Candidate Measures · p. 16

F LLM-Judge Baseline Rubric

  • The GPT-4o B5 baseline scores four rubric dimensions on 1–5 scales and selects the top 1,024 trajectories; estimated judging cost is roughly $2,750 for 38,382 eligible trajectories and $18.6K for 258,134 unique trajectories.The rubric dimensions are planning, grounding, reflection, and efficiency; the judge does not see the reference patch or held-out test outcome.Source: F LLM-Judge Baseline Rubric · p. 16

G Length-Controlled Pair-AUC

  • Within the 25–40-step band, composite TQS remains perfectly separating, while OEC, IAC, and PED each remain above 0.96 pair-AUC and PSN falls to 0.839.The PSN drop indicates scope narrowing is partly entangled with trajectory length, but the composite score is not driven by that component alone.Source: G Length-Controlled Pair-AUC · p. 16, Table 11 · p. 16

H Tokenized-Length Cutoff

  • The 32,768-token cap leaves 38,382 of 155,144 passing trajectories eligible (24.7% retention), and the discarded tail is dominated by long trajectories, especially from SWE-Smith.The cap is a fixed-budget SFT choice; longer trajectories may contain useful recovery behavior but require longer-context or sequence-parallel training to imitate faithfully.Source: H Tokenized-Length Cutoff · pp. 16–17, Table 12 · p. 17

I Sensitivity to Component Weights

Moderate perturbations to the hand-set component weights leave most of the Top-1024 selection unchanged, although this does not establish that the weights are globally optimal.

I Sensitivity to Component Weights

  • The main experiments use fixed weights of 0.40 for information focus, 0.25 each for intent–action alignment and policy dynamics, and 0.10 for the DPI guardrail.No downstream pass@1 result was used to tune the weights; the sensitivity test perturbs wOEC, wIAC, and wDPI and rebuilds the Top-K subset.Source: I Sensitivity to Component Weights · p. 17
  • 93–94% overlap for OEC and IAC perturbations and over 98% for DPI perturbations indicate local stability of the Top-1024 selection.Figure 8 visualizes overlap with the main selection.Source: I Sensitivity to Component Weights · p. 17, Figure 8 · p. 17
  • The analysis shows local stability around the chosen weights, but does not establish that they are globally optimal.Small perturbations leave most of the selected training set unchanged.Source: I Sensitivity to Component Weights · p. 17
Loading 2610.13152v1…