PreciseCoder: Faithful and Generalizable Debugging Beyond Unit-Test Rewards

1 Diagnosis

• Unit-test-only RL raises pass rates without restraining unnecessary edits and increasingly leads models to modify code that is already correct.The paper audits training rollouts and reward-term correlations to examine this behavior.Source: §1 · pp. 1–2

2 Fix

• Semantically verified edit precision and recall, combined as F1, augment the unit-test reward and correlate with both test outcomes and edit size.This combination is designed to improve edit fidelity without sacrificing pass rate.Source: §1 · pp. 1–2
• 7–15 to 3 average edited lines per task, with pass rates on par with unit-test-only GRPO across 4B, 9B, and 27B models.PreciseCoder also raises edit precision by 24–29% over base models and 26–35% over unit-test-only GRPO.Source: §1 · p. 2

3 Generalization

• Trained only on single-line bugs, PreciseCoder generalizes to multi-line and repository-level repairs, resolves more SWE-bench Verified issues with smaller patches, and breaks fewer already-fixed repositories.In agentic debugging, a 9B model reaches 67.4% edit precision at a 56.3% pass rate, best among the reported 9B models on both metrics.Source: §1 · p. 2
• A 9B PreciseCoder reasoning trace raises frontier models’ edit precision by 4.6–12.4 points without additional training.Trace analysis associates this transfer with reasoning about individual lines, naming buggy lines, and weighing edits.Source: §1 · pp. 2–3

PreciseCoder

• PreciseCoder incorporates semantically verified, localized edits into RL training for generalizable debugging fidelity.The paper presents it as the first approach to do so.Source: §2 · p. 3

Research context

• Program-repair research documents test overfitting, while functional correctness alone does not capture all qualities developers value.Prior work also reports unnecessary changes when code reviewers hallucinate bugs.Source: §2 · p. 3

Existing remedies

• Existing over-editing remedies penalize edit size or use localization and patch-refinement approaches, but edit-size rewards may trade test-passing performance for debugging fidelity.Examples include EA-GRPO, Zhu et al. (2026a), RECAP (Luo et al., 2026), and localization-first methods (Xia et al., 2024; Chen et al., 2025; Yang et al., 2025b).Source: §2 · p. 3

3 METHOD

• The task is a faulty program, specification, test suite, and reference repair; the policy emits a revised program whose difference from the input is the predicted edit.The standard RL reward is whether the revised program passes the test suite.Source: §3 · p. 3

3.1 FIDELITY CONCERNS WITH UNIT-TEST-ONLY GRPO

• 3.94 to 8.28 mean edited lines on the training set as unit-test-only GRPO continues periodically encouraging larger edits after pass rates begin to plateau.During earlier training, edit size tracks pass rate as both rise and fall.Source: §3.1 · p. 4
• 20.5% no-op rate on 200 bug-free validation programs in the last three evaluations, down from 30.8% at training start, shows unit-test-only GRPO also modifies correct code.Training does not show the model any correct programs.Source: §3.1 · p. 4

3.2 HOW DIFFERENT REWARD TERMS CORRELATE

• −ℓ correlates weakly with the pass indicator (0.13 on 4B; 0.14 on 9B), while precision correlates with −ℓ (>0.81) and recall with the pass indicator (>0.71).The method scores predicted edit blocks against reference faults; recall measures repaired faults, while precision measures the share of edited lines needed to repair them.Source: §3.2 · pp. 4–5

3.3 ALGORITHM

• F1 combines precision with recall gated on the pass indicator, correlating with both the pass indicator (>0.62) and −ℓ (>0.82).PreciseCoder mixes F1 with the unit-test reward and separately standardizes each term within the group before combining them.Source: §3.2 · p. 5, §3.3 · p. 5

4.1 SETUP

• Training uses 5,274 PDB-Single-Full tasks and tests on 1,205 held-out tasks with no shared base problem, evaluating pass rate, precision, recall, and edited lines.Correct-code behavior is tested separately on 1,512 ground-truth programs; the backbones are Qwen3.5-4B, Qwen3.5-9B, and Qwen3.6-27B.Source: §4.1 · p. 5

4.2 MAIN RESULTS

• 3.0 edited lines on Qwen3.5-4B, down from 13.8, keeps pass rate on par with unit-test-only GRPO; PreciseCoder similarly reduces editing on Qwen3.5-9B.PreciseCoder ranks in the top two on every metric; Table 2 summarizes its pass-rate, edit-size, and correct-code results.Source: §4.2 · p. 5, Table 2 · p. 6
• GPT-5.6 Sol has the highest pass rate at 86.1%, but its precision is 17.7% below PreciseCoder-27B’s 79.2% and it edits 1.5 more lines on average.PreciseCoder-27B also outperforms its GRPO counterpart, DeepSeek-V4.1-Flash, and Qwen3.8-Flash-Next on all measures.Source: §4.2 · p. 6

4.3 BUG HALLUCINATION ON CORRECT CODE

• On correct code, PreciseCoder raises no-op rates to 48.0%, 51.6%, and 45.8% at 4B, 9B, and 27B, while unit-test-only GRPO lowers them.No-op rate and edit precision are highly correlated across 4B–27B models (r = 0.87).Source: §4.3 · p. 6

4.4 REWARD ABLATIONS

• Precision alone over-restrains and recall alone under-restrains; only their gated combination with the unit-test reward keeps pass rate on par.Precision-focused ablations reach 69.0–70.1% precision with 2.5–2.6 lines edited, but pass 3–5% fewer tasks than PreciseCoder.Source: §4.4 · pp. 6–7, Table 3 · p. 7

5.1 GENERALIZATION TO COMPLEX PROGRAM REPAIRS

• On PDB-Wild, precision rises 20.3, 16.7, and 20.2 points over base at 4B, 9B, and 27B, while mean edits shrink from 37.8, 33.9, and 18.4 lines to 10.8, 12.0, and 6.8.PreciseCoder also raises pass rate by 10–13 points at every size and outperforms EA-GRPO across sizes in pass rate and precision.Source: §5.1 · p. 7
• On SWE-Bench Verified, PreciseCoder resolves more issues with smaller patches; its 9B model resolves 32.7% versus 28.1% for GRPO, with 29% versus 58% excess changed lines.With the gold patch already applied, PreciseCoder-27B breaks 8.9% of repositories versus 11.0% for base and 13.1% for GRPO.Source: §5.1 · p. 8, Figure 5 · p. 8
• In five-turn agentic editing, single-turn-trained PreciseCoder-9B reaches 52.6% precision; agentic RL training raises this to 67.4% at a 56.3% pass rate.The agentically trained model is the best 9B model on both measures in agentic debugging.Source: §5.1 · p. 8, Figure 6 · p. 8

5.2 WEAK-TO-STRONG TRANSFER

• A PreciseCoder-9B reasoning trace raises edit precision by 4.6–12.4 points across five stronger receivers, with pass-rate changes from −1.5 to +4.3 points and no shared weights or training.Prefilled into its own base model, the trace recovers 73% of the training pass-rate gain and 60% of the precision gain, reducing edits from 13.7 to 6.5 lines.Source: Table 4 · p. 8, §5.2 · p. 9

5.3 WHAT CHANGED IN THE THINKING

• PreciseCoder traces more often judge individual lines: they hedge more and echo instructions less than GRPO, while quoting every buggy line correlates with 4–19% higher recall.The traces also propose more bug hypotheses on correct programs, and no-op rates rise from 21% to 68% across the lowest-to-highest hypothesis quartiles.Source: §5.3 · pp. 9–10, Figure 7 · p. 10, Figure 8 · p. 10

6 DISCUSSION

• PreciseCoder combines semantically verified edit precision and recall with unit-test outcomes, retaining pass rates while improving debugging fidelity across repository repair and already-fixed repositories.The discussion also reports that a 9B model’s reasoning trace raises frontier-model edit precision without transferring its weights.Source: §6 · p. 10
• A reference-checked action reward gives no credit to correct but unnecessary actions, and could address side effects in long-horizon agents; defining the reference for repositories remains future work.Lower damage on already-fixed repositories is described as an initial sign of this broader potential.Source: §6 · p. 10
• The work uses public datasets, benchmarks, and open-weight models, executes generated code in a sandbox, and recommends human review because subtle errors may be harder to spot in precise patches.The paper reports no human subjects or known conflicts of interest.Source: ETHICS STATEMENT · p. 10
• The authors describe the reward, optimizer, training setup, prompts, hyperparameters, and base-task data splits, and state that code, splits, and checkpoints will be released.Variants of the same problem do not cross data splits.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 PASS/FAIL ORDERING UNDER DECOUPLED NORMALIZATION

• Decoupled normalization preserves passing-over-failing completion order below a worst-case weight bound; positive batch rescaling preserves the order, and positive F1/unit correlation makes the worst case rare.F1 is constant on fewer groups than the unit-test term: 17–24% versus 43–46% of groups of six base-model samples.Source: §A.1 · p. 16, §A.2 · p. 16

A.2 COLLAPSED TERMS IN GROUPS

• F1 restores ranking when unit-test rewards are constant: it rewards edit minimality in uniformly passing groups, partial repairs in failing groups, and breaks ties within pass/fail sides in mixed groups.Reward advantages collapse on 65% of groups under unit-test-only reward, 48% under EA-GRPO, and 39% under PreciseCoder.Source: §A.2 · pp. 16–17

B.1 GRPO + CLEAN ROWS

• 73.7% and 71.8% no-op rates with 7.9% damage at both sizes come from clean-row training, but pass rates are 50.6% and 61.3%, below unit-test-only GRPO’s 55.1% and 63.9%.Clean rows alone still raise precision over unit-test-only GRPO by 23.9 points at 4B and 13.4 at 9B, while cutting edits by 10.0 and 6.0 lines.Source: §B.1 · p. 17, Table 5 · p. 17

B.2 OUT-OF-DISTRIBUTION GENERALIZATION

• PreciseCoder beats its base model on all five unseen code benchmarks at both sizes; at 4B it beats unit-test-only GRPO on all but DebugBench, while at 9B GRPO leads on four by at most 1.5 points.PreciseCoder-9B leads unit-test-only GRPO on LiveCodeBench by 5.3 points.Source: §B.2 · pp. 17–18, Table 6 · p. 18

B.3 REWARD-WEIGHT SEARCH

• Figure 9 shows that at λ=1/6, edit precision rises from 33.4% to 63.6% and F1 from 39.9% to 64.4%, at a 2.9-point pass-rate cost; λ=1/3 is used.Across λ values from 1/6 to 5/6, each of the three metrics varies by at most 2.5 points.Source: Figure 9 · p. 18, §B.3 · p. 18

B.4 PROMPTING ANALYSIS ON GT DEBUG TASK

• The abstain-or-fix prompt raises every model’s no-op rate by 14–44 points and lowers damage, while PreciseCoder beats base and EA-GRPO on both rates at 4B and 9B.At 9B, PreciseCoder narrows the no-op gap to clean-row training from 20.2 to 4.3 points, with 7.4% damage versus 8.0%.Source: §B.4 · p. 18, Table 7 · p. 19

B.5 CHAIN-OF-THOUGHT TRANSFER FROM EVERY 9B DONOR

• With Qwen3.6-27B as receiver, PreciseCoder-9B traces raise precision by 10.1 points on average and pass rate by 2.8 points, while the base 9B trace costs 7.4 pass-rate points.Only trained donors preserve or raise receiver pass rate; the largest reported PreciseCoder pass-rate gain is 4.3 points for seed 42.Source: Table 8 · p. 19, §B.5 · p. 19

B.6 DETAILED LINGUISTIC ANALYSIS ON THE THINKING

• The analysis uses lexical categories and code-line grounding, with performance relationships measured as within-model associations rather than causal effects.Grounding checks whether each buggy line appears verbatim in the prose after whitespace normalization; lexical categories were fixed before correlations were computed.Source: §B.6 · pp. 19–20
• On buggy inputs, both rewards produce similarly long traces, but PreciseCoder adds hedging and reduces minimal-edit talk, instruction echo, and claims that the prompt says a bug exists.Prose-to-description TF-IDF cosine is 0.39–0.41 for all six local models, and 2–3% of prose 5-grams restate the prompt across models.Source: §B.6 · p. 20, Table 9 · p. 20
• Frontier models retain distinct linguistic styles: GLM-5.2 uses “let me” 17 times per 1,000 words, while DeepSeek-V4-Pro asks 51 questions per 1,000 words.The Jensen–Shannon divergence from GLM-5.2 grows under either reward for 9B models.Source: §B.6 · p. 20
• On correct inputs, trained models often continue a bug hunt; PreciseCoder-9B proposes 28 hypotheses and repeats its most frequent sentence 195 times, while leaving clean code alone more often when thinking reaches its budget.PreciseCoder-9B’s no-op rate is 52% at the cap versus 14% when it stops early; the unit-test-only 9B model is 34% versus 1%.Source: §B.6 · pp. 20–21, Table 10 · p. 21
• For PreciseCoder-9B, no-op rates rise from 21% to 68% between the lowest and highest hypothesis-count task quarters, while the unit-test-only 9B model stays near 30%.PreciseCoder-4B instead loops least and returns an empty answer on 6.7% of programs when thinking exhausts its answer budget.Source: §B.6 · p. 21

• Figure 11 compares how unit-test-only GRPO and PreciseCoder change 9B thinking features relative to the base model, in within-triple standard deviations with 95% confidence intervals.Source: Figure 11 · p. 22
• Figure 12 plots PDB Test edit precision against pass rate, with bubble area for edited lines and arrows linking each base model to its PreciseCoder model.It shows the PDB Test columns of Table 2.Source: §B.7 · p. 21, Figure 12 · p. 22
• Figure 13 depicts chain-of-thought transfer on PDB Test, including a PreciseCoder trace given to its own base model and to larger receivers.It corresponds to the transfer results in Table 4.Source: §B.7 · p. 21, Figure 13 · p. 23
• Figure 14 compares GT Debug under minimal-edit prompts (circles) and abstain-or-fix prompts (triangles), with upward and rightward directions indicating better results.It shows the prompt ablation in Table 7.Source: §B.7 · p. 21, Figure 14 · p. 23

C.1 HYPERPARAMETERS

• All reinforcement-learning runs share the upper-block settings, enable thinking, enforce a thinking budget in training and evaluation, and save validation checkpoints every 25 steps.Table 11 summarizes the training and evaluation hyperparameters, including settings that vary by model size and trace-transfer configuration.Source: §C.1 · p. 24, Table 11 · p. 25

C.2 BASELINES

• Baselines comprise self-distilled rejection-sampling SFT and DPO, gated edit-size-penalty EA-GRPO, and two-stage RECAP refinement.SFT ranks passing samples by edit precision at 4B and edit F1 at 9B; EA-GRPO penalizes passing rollouts according to their relative edit size only after a group-accuracy gate activates. RECAP uses a base model to propose a fix and a separately fine-tuned refiner to produce a minimal repair.Source: §C.2 · p. 24

D.1 MINIMAL-EDIT PROMPT

• The minimal-edit prompt instructs the model to fix only bugs, make minimal changes, preserve correct formatting, and return a self-contained corrected implementation.The same prompt is used for training and most single-turn evaluations, and the unit tests are never shown to the model.Source: §D.1 · pp. 25–26, D PROMPTS · p. 25

D.2 REFINE PROMPT (RECAP)

• The RECAP refine prompt asks the model to keep necessary candidate changes, remove unnecessary ones, correct incomplete fixes, and return an already-minimal fix unchanged.The candidate is the base model’s output under the minimal-edit prompt.Source: §D.2 · p. 26

D.3 ABSTAIN-OR-FIX PROMPT

• The abstain-or-fix prompt first asks whether code contains bugs, requiring unchanged output when it does not and minimal repairs otherwise.It retains the minimal-edit prompt’s prohibitions against generating a new solution, reformatting correct lines, and editing comments.Source: §D.3 · pp. 26–27

D.4 CHAIN-OF-THOUGHT TRANSFER

• Chain-of-thought transfer gives a receiver the donor’s thinking trace, either closed before its answer or left open for continued reasoning within a budget.When assistant-turn prefill is unavailable, the trace is appended to the user message instead.Source: §D.4 · p. 27

D.5 AGENT TOOL PROMPT

• Agentic evaluation runs each task in a five-turn tool loop, applies one tool call per turn, and scores the current file at submission or timeout.The harness disables run_tests in reported runs; view displays lines, while str_replace_editor requires an exact unique match and leaves the file unchanged on failure.Source: §D.5 · pp. 27–28

• Training and evaluation are in Python; extending the method to another language requires reference fixes and a test harness, and this extension was not tested.Source: E LIMITATIONS · p. 29
• Every training task needs a reference repair, which can come from a benchmark, a human-written patch, or a model-generated fix that passes unit tests.Source: E LIMITATIONS · p. 29
• Edit scoring adds about 5–7% to unit-test-only training wall-clock time, though caching and larger-scale generation or updates amortize the cost.Source: E LIMITATIONS · p. 29