Source-linked AI summary
Coaching Qwen3 Coder 30B to Think Like a CodeClash Arena Agent
Ivy Ning Zhang
TL;DR
Weak coding agents remain difficult to improve for long-horizon tasks that require repeated feedback and strategic edits. The paper studies targeted posttraining of Qwen3 Coder 30B using structured teacher trajectories and trajectory-quality weighting. In CODECLASH BattleSnake, the final 6-turn ReAct model substantially outperformed the base model and clearly exceeded TQ-SFT in tournament play.
Problem
The paper asks how to improve weak coding agents’ long-horizon behavior, which standard instruction tuning does not reliably teach from arena trajectories.
Source: §1 · p. 1, §2.3 · p. 2Method
The authors compare ReAct SFT, which structures teacher trajectories as [Obs][Thought][Act], with TQ-SFT, which upweights safer modify-then-check trajectories.
Source: §1 · p. 2, §4.3 · p. 4Results
In CODECLASH BattleSnake, the final 6-turn ReAct model substantially outperformed the base model and clearly exceeded TQ-SFT in tournament play.
Source: §7 · p. 7Takeaways & Limitations
The results suggest that structured posttraining can transfer a feedback-driven action policy for open-weight coding agents, though robustness gaps remain.
Source: §7 · pp. 7–8The study focuses on Qwen3 Coder 30B in CODECLASH BattleSnake and identifies substantial remaining robustness gaps.
Source: §7 · p. 8
Abstract
from arXiv · showhide
Large language model coding agents have recently become useful for software tasks, but weaker or open-weight agents still struggle to reliably interpret user intent and execute complex multi-step workflows. This gap is especially visible in long-horizon settings, where an agent must repeatedly inspect prior outcomes, diagnose failure, and choose the next code edit under interaction constraints. It motivates a natural question: what can we do to improve the thinking process of a weak code agent? We study this question in CodeClash, a code-arena benchmark where the original work evaluates 8 commercial coding agents across 6 arenas through multi-round tournaments. Since Qwen3 Coder Plus ranks last among them, we take the open-weight Qwen3-Coder-30B as a case study and investigate how to improve it with distilled knowledge from stronger agents. Our analysis shows that Qwen3-Coder-30B is not well optimized for arena-style interaction: it frequently produces syntax and protocol-breaking errors and exhibits weak strategic adaptation across rounds. These failures are difficult to correct with vanilla instruction tuning alone, since offline SFT cannot directly verify whether a generated action is valid or beneficial. To address this, we propose ReAct SFT, which rewrites teacher trajectories into explicit [obs][thought][act] chains, and trajectoryquality weighted SFT, which reweights samples to encourage post-edit checking. ReAct SFT substantially improves strategic behavior, and our fine-tuned model outperforms the original Qwen3 Coder Plus in tournament evaluation.
1 Introduction
The paper asks how to improve weak coding agents’ long-horizon behavior, which standard one-shot coding benchmarks do not fully capture. In CODECLASH, the authors study Qwen3 Coder 30B and propose targeted supervision to address its arena-interaction weaknesses.
1 Introduction
- Standard coding benchmarks emphasize one-shot generation or issue resolution, motivating a question about improving weak agents’ long-horizon behavior.The motivating gap is agents’ inconsistent interpretation of instructions and execution of complex multi-step workflows.Source: §1 · p. 1
- Qwen3 Coder 30B frequently makes syntax and protocol-breaking errors and underuses log inspection, outcome analysis, and post-edit validation.The paper uses the open-weight model as a posttraining case study, with Claude Sonnet 4.5 trajectories as the ReAct SFT teacher source.Source: §1 · pp. 1–2
- ReAct SFT produces the strongest competitive gains, while TQ-SFT mainly reduces invalid submissions without matching its tournament improvement.ReAct SFT structures teacher trajectories as [Obs][Thought][Act]; TQ-SFT upweights safer modify-then-check behavior.Source: §1 · p. 2
2 Related Work
Related work motivates CODECLASH as a setting for repeated-feedback coding, connects ReAct-style supervision to sequential decision-making, and identifies limits of instruction tuning for transferring arena-specific reasoning.
2.1 Code arenas and interactive software engineering benchmarks
- CODECLASH evaluates iterative repository editing through multi-round competition, unlike benchmarks focused on local generation or issue resolution [1] [4].This setting aligns with the paper’s goal of improving agent behavior under repeated feedback and competitive pressure.Source: §2.1 · p. 2
2.2 Reasoning and acting supervision for language agents
- ReAct structures outputs as reasoning followed by task-specific actions, a relevant approach when arena agents must interpret prior outcomes and choose subsequent edits [5].The paper adapts this perspective by rewriting teacher trajectories into explicit [Obs][Thought][Act] supervision.Source: §2.2 · p. 2
2.3 Instruction tuning and its limits in long horizon coding
- Instruction tuning does not uniformly transfer skills [7] [8], and preliminary experiments suggest vanilla tuning can teach response format without the arena’s observation-reasoning-action loop.This motivates comparing ReAct supervision for competitive reasoning with trajectory weighting for protocol reliability.Source: §2.3 · p. 2
3 Task Setting
The task is long-horizon coding in CODECLASH BattleSnake, where agents edit a private repository and compete across rounds. Logs and results feed subsequent decisions, while each interaction follows a strict single-action protocol.
3 Task Setting
- In CODECLASH, agents iteratively modify private repositories and are evaluated through multi-round tournaments in an executable arena.BattleSnake alternates between a budgeted terminal-based edit phase and a competition phase.Source: §3 · p. 2
- Competition logs and round results return to the repository as feedback, and each step requires exactly one bash command with a THOUGHT section.The agent receives the execution result before continuing.Source: §3 · p. 3
- The study targets execution fragility and weak strategic adaptation, including limited log use, post-edit validation, and use of previous outcomes.The objective is to improve both reliability and competitive reasoning under repeated feedback.Source: §3 · p. 3
4 Approach
The approach compares standard SFT baselines with two targeted signals: explicit observation-reasoning-action supervision and quality-based weighting toward safer edits. Shared training treats arena turns as conversational trajectories and computes loss on assistant outputs.
4 Approach
- The posttraining pipeline starts with self-play and teacher-distilled SFT baselines, then adds ReAct SFT and TQ-SFT as targeted supervision signals.ReAct makes the observation-reasoning-action structure explicit; TQ-SFT favors safer modify-then-check behavior.Source: §4 · p. 3
4.1 Shared SFT Setup
- Each CODECLASH trajectory is treated as a multi-turn conversational instance, with each assistant response conditioned on the history through the current user observation.The user observation and corresponding assistant output may be a THOUGHT plus one bash action block.Source: §4.1 · p. 3
4.2 ReAct SFT
- ReAct SFT rewrites teacher trajectories into [Obs][Thought][Act] targets, preserving the original bash command while making the decision process explicit.Claude Sonnet 4.5 annotates turns using prior execution output, current context, original THOUGHT, and original command; the reasoning is rewritten to summarize feedback, state the step’s strategic purpose, and name the operation.Source: §4.2 · pp. 3–4, Figure 1 · p. 4
4.3 Trajectory Quality Weighted SFT (TQ-SFT)
- TQ-SFT derives trajectory-quality scores from teacher action sequences and uses them to weight samples, favoring safer modify-check-submit behavior over unchecked edits.The score is based on behavior categories such as MODIFY_MAIN, CODE_CHECK, SUBMIT, STRATEGY_READ, and STRATEGY_ANALYSIS; unlike ReAct SFT, it does not explicitly teach competitive reasoning.Source: §4.3 · p. 4
5 Experiments
Experiments compare three posttraining approaches for Qwen3 Coder 30B in CODECLASH BattleSnake, assessing competitive outcomes and reliability across tournament rounds. Training data and supervision structure vary across approaches, while evaluation uses the same arena scaffold.
5.1 Experimental Setup
- The study compares vanilla self-play SFT, ReAct SFT, and TQ-SFT from the same Qwen3 Coder 30B base in BattleSnake.All evaluations use the same scaffold and original CODECLASH tournament setup.Source: §5 · p. 5, §5.1 · p. 5
5.2 Training Data
- Training data ranges from self-play and recorded teacher trajectories to rewritten Claude Sonnet 4.5 steps and quality-weighted trajectories from five stronger agents.Vanilla SFT includes about 40% valid self-play round trajectories, 1,005 Gemini 2.5 Pro trajectories, and 975 Qwen3 Coder Plus trajectories. ReAct uses up to 430 rewritten trajectories; TQ-SFT uses 4,905 trajectories annotated with rule-derived action labels and quality scores.Source: §5.2 · p. 5
5.3 Evaluation
- Evaluation measures non-tie round win rate across 1,000 simulator runs, tournament outcomes over 15 rounds, and reliability outcomes.Reliability measures include invalid submission rounds and round collapse behavior.Source: §5.3 · p. 5
5.4 Experimental Details
- ReAct context windows vary across 2, 4, 6, and 8 turns, while TQ-SFT uses 6; optimization is held fixed across variants.Training uses sliding windows of assistant turns with corresponding observations to accommodate long trajectories and limited GPU memory.Source: §5.4 · p. 6
5.5 Preliminary Baselines
- Vanilla trajectory SFT sometimes improves self-play win rate but remains structurally fragile, while teacher-data SFT often collapses and does not reliably improve strategy.Reported failures include syntax corruption and removal of the BattleSnake starter block; self-play SFT may adapt to trajectory surface form without reliably improving strategy or inference-time safety.Source: §5.5 · p. 6
6 Results and Analysis
ReAct SFT’s final 6-turn model wins decisively against the base model and Qwen3 Coder Plus, though it remains below Claude 4.5. The results and discussion contrast its strategic adaptation with TQ-SFT’s reliability-oriented behavior.
6.1 Main Tournament Results
- Among early context-length ablations, ReAct 4-turn performs best; 2 turns preserve too little history, while 8 turns are too long and noisy.These ablations use an initial rewritten subset of about 150 trajectories.Source: §6.1 · p. 6
- TQ-SFT improves some robustness behaviors but underperforms ReAct 4-turn in competition; its self-play variant reduces invalid submissions from roughly 40% to 25% yet fails more often in tournaments.The authors suggest imitation-based finetuning may increase deployment-time failures through behavioral drift.Source: §6.1 · p. 6
- ReAct 6-turn decisively beats Qwen3 Coder 30B base and Qwen3 Coder Plus, remains below Claude 4.5, and has fewer abnormal collapse rounds than earlier variants and TQ-SFT.The final model was trained on an expanded rewritten corpus of up to 430 trajectories.Source: §6.1 · pp. 6–7, Table 3 · p. 7
6.2 Discussion
- ReAct’s competitive gains are associated with an interaction-process shift: agents revisit results, inspect logs, and tie edits to observed failure modes.The authors interpret this as transferring part of stronger agents’ observation-reasoning-action loop.Source: §6.2 · p. 7
- TQ-SFT upweights safer modify-then-check trajectories, improving reliability, but does not directly teach strategically useful edits and can increase caution without much adaptation.Behavioral analysis contrasts ReAct’s round-aware diagnosis and testing with TQ-SFT’s checking and analysis behaviors.Source: §6.2 · p. 7
7 Conclusion and Future Work
The final 6-turn ReAct model substantially outperformed the base model and clearly exceeded TQ-SFT in tournament play. The authors conclude that transferring a feedback-driven action policy is promising, while noting remaining robustness gaps and future work on broader models and supervision.
7 Conclusion and Future Work
- The final 6-turn ReAct model substantially outperformed the base model and clearly exceeded TQ-SFT in tournament play.The model was trained after expanding the rewritten corpus beyond the initial ablation subset.Source: §7 · p. 7
- The results suggest that sequential software tasks require a feedback-driven action policy connecting observations, reasoning, and edits over repeated rounds.The authors say standard instruction tuning or reliability-oriented filtering alone is not enough.Source: §7 · pp. 7–8
- ReAct SFT shifts behavior toward round-aware diagnosis and testing, moving closer to Claude 4.5, while TQ-SFT mainly increases testing and scripted analysis associated with reliability.Table 4 reports percentages of assistant bash actions in selected categories; its columns do not sum to 100%.Source: Table 4 · p. 8
- Substantial robustness gaps remain; future work includes comparing similar-scale open-weight models and studying stronger supervision that combines strategic reasoning with explicit validation.Source: §7 · p. 8
A.1 Failure Analysis of Vanilla and Self-Play SFT Baselines
Vanilla and self-play SFT agents remain vulnerable to invalid submissions and structural code corruption, despite gains over the base model in some rounds. Limited diagnostic effort and high-risk edits motivate explicit validation and diagnostic safeguards.
A.1 Failure Analysis of Vanilla and Self-Play SFT Baselines
- Both self-play SFT and base models still produce multiple 0v0 all-tie rounds caused by invalid submissions or service failures.The self-play SFT model improves over the base model in several rounds.Source: §A.1 · p. 9
- Frequent self-play failures include accidental removal of startup structure and structural corruption, while large-range edits can still introduce invalid code despite a safeguard.Examples include removing run_server, returning outside a function, or breaking the def move signature; a safeguard reduces startup-block removal but does not eliminate instability.Source: §A.1 · p. 9
- Qwen-based agents perform fewer structured diagnostic actions and often make large edits without sufficient intermediate validation, increasing syntax and runtime-error risk.Full overwrites and truncate-and-append patterns can disrupt code structure; Figure 2 relates error-checking and diagnosis behavior to BattleSnake win rate.Source: §A.1 · p. 9, Figure 2 · p. 9
- The authors recommend behavior-level safeguards such as pre-submit py_compile checks and startup-block validation, alongside more systematic diagnostic procedures.Instruction tuning alone does not ensure runtime validity within the deployment scaffold.Source: §A.1 · p. 9