PaperlayerView exact revision on arXiv

Source-linked AI summary

No One-Size-Fits-All: Adaptive Code Editing with Feature-Based Strategy Selection

Jun Wan, Zhongxin Liu, Dajun Chen, Wei Jiang, Yong Li, Xiaoxue Ren

arXiv:2610.22881v1cs.SE

TL;DR

Code-editing systems commonly apply fixed strategies despite diverse tasks, and systematic evidence about strategy-task relationships has been limited. The paper compares strategies on CEBench, identifies task features associated with their effectiveness, and proposes Ace to select strategies adaptively. Ace improves Pass@1 by 21.7% while reducing token consumption by 27.3% versus the best fixed strategy, search-replace.

  • Problem

    Existing code-editing systems often use fixed strategies across diverse tasks, while systematic study of how strategies relate to task features remains limited.

    Source: §1 · p. 3

  • Method

    The paper compares three strategies on CEBench and proposes Ace, which uses a fine-tuned LLM to select a strategy from task features and provide rationales.

    Source: §1 · p. 3

  • Results

    Strategies have distinct advantages across task characteristics, and Ace improved Pass@1 by 21.7% while reducing token consumption by 27.3% versus search-replace.

    Source: §1 · p. 3

  • Takeaways & Limitations

    No single code-editing strategy is universally optimal; strategy effectiveness varies with task features.

    Source: §1 · p. 3

    Feature extraction and evaluation used Python code from CEBench, so performance may decline on non-Python languages and the features may not capture distinct tasks such as code completion or competitive programming.

    Source: §8.3 · p. 20

Abstract

from arXiv · show

Large language models have shown strong potential in automating code editing (CE) tasks, yet most existing systems rely on a fixed CE strategy (i.e., one-step, search-replace, and plan-apply) across diverse scenarios. In practice, CE tasks vary widely in code context, edit locality, and semantic complexity, causing no single strategy to perform consistently well. Moreover, systematic studies on the relationship between strategies and CE tasks remain scarce, limiting further optimization of LLM-based code editing strategies. To address this gap, we construct CEBench, comprising 500 real-world CE tasks curated from SWE-bench. Then we compare three representative CE strategies (one-step, search-replace, plan-apply). Our study reveals two key findings: (1) each strategy exhibits distinct advantages for specific CE tasks, with no universal optimality; (2) strategy effectiveness is significantly influenced by four core task features: total lines of code, modification type, modification scope, and task type. Motivated by these findings, we propose Ace, an adaptive code editing framework that dynamically selects optimal CE strategies based on task features. Unlike classifier-based or instruction-prompting approaches, Ace employs a fine-tuned LLM that learns feature-strategy mappings from empirical data to recommend strategies with interpretable rationales. Extensive experiments on our CEBench-verified show that Ace outperforms state-of-the-art baselines: it boosts Pass@1 by 21.7% while reducing token consumption by 27.3% compared to the best baseline (AutoCodeRover). Our strategy selector achieves 72.6% selection accuracy with 69.9% rationale accuracy (Cohen's Kappa = 0.807), providing transparent and trustworthy decision-making for developers.

1 Introduction

Code editing is a common but challenging activity, and the three major LLM-based strategies trade off correctness, efficiency, and implementation demands. The study finds that strategy effectiveness depends on task features, motivating Ace’s feature-based selection and evaluation against existing approaches.

1 Introduction

  • Code editing modifies existing source code, but it remains challenging because even small changes require understanding semantics and dependencies and can introduce unintended side effects.The introduction notes that developers spend over six hours per week on code editing tasks.Source: §1 · p. 2
  • One-step rewrites code, search-replace generates and applies diffs, and plan-apply separates planning from application, with distinct efficiency, precision, and model-capability trade-offs.Plan-apply reduces planning token use but adds system complexity and demands more from the applying model; weaker models can produce incomplete or inaccurate edits or unauthorized changes.Source: §1 · pp. 2–3
  • The study compares one-step, search-replace, and plan-apply using CEBench, a 500-task benchmark derived from SWE-bench, to examine strategy effectiveness and influential task features.Source: §1 · p. 3
  • No CE strategy is universally optimal: each has distinct advantages, and effectiveness is significantly influenced by total lines of code, modification type, modification scope, and task type.Source: §1 · p. 3
  • Ace uses a fine-tuned LLM to extract task features, recommend one of the three strategies with interpretable rationales, and guide the subsequent code edit.Source: §1 · p. 3
  • Ace improved Pass@1 by 21.7% and reduced token consumption by 27.3% versus the best-performing fixed strategy, search-replace, on CEBench-verified.Its selector achieved 72.6% selection accuracy and 69.9% rationale accuracy (Cohen’s Kappa = 0.807).Source: §1 · p. 3

2 Background and Related Work

The paper frames code editing as modifying existing code in response to an edit request, then surveys rule-based and LLM-based approaches. Its related-work discussion highlights fixed-strategy limitations that motivate adaptive selection.

2.1 Code Editing Problem Formulation

  • Given source code and an edit request, code editing identifies locations and changes, then applies an editing function to produce updated code.Traditional approaches use manually defined rules or static analysis, while LLM-based techniques can perform the editing function.Source: §2.1 · p. 4

2.2 Code Editing Technique

  • The field has moved from rule-based and statistical methods toward LLM-driven approaches focused on flexibility, generalization, and context awareness.Source: §2.2 · p. 4

2.2.1 Non-LLM-Based Code Editing.

  • Rule-based methods handled constrained syntax errors or simple statement changes, but their manually defined rules limited flexibility and generalization.Pretrained models later demonstrated stronger valid-patch generation than traditional methods.Source: §2.2.1 · p. 4

2.2.2 LLM-Based Code Editing.

  • LLM-based methods include one-step rewriting, search-replace diff application, and plan-apply staging, each with different trade-offs in localization, efficiency, and editing precision.Search-replace is sensitive to search errors and may miss inter-code dependencies; plan-apply separates planning from application but adds pipeline complexity and intermediate token overhead.Source: §2.2.2 · pp. 4–5
  • Because existing strategies use fixed architectures despite diverse task requirements, Ace dynamically selects strategies based on the task and source code.Source: §2.2.2 · p. 5

3 Empirical Study Setup

The empirical study compares representative editing strategies and analyzes which task features relate to their performance. Its setup defines the evaluation measures, adapts techniques to a controlled setting, and constructs CEBench with strategy labels.

3.1 Research Questions

  • RQ1 measures differences in correctness and token efficiency among three strategies, while RQ2 identifies task features associated with their relative effectiveness.RQ1 uses Claude-Opus-4.1 for all methods on CEBench; RQ2 extracts features from successfully solved cases and validates them through correlation analysis.Source: §3.1 · p. 5

3.2.1 Techniques of Different CE Strategies.

  • The comparison isolates code editing from cross-file repository context and adapts CodePlan, AutoCodeRover, and FuseApply by retaining their core logic while removing repository-specific components where needed.CodePlan retains intra-file context extraction, AutoCodeRover retains AST-based search and patch retries, and FuseApply requires no adjustment for the single-file scenario.Source: §3.2.1 · p. 6

3.2.2 Feature Analysis.

  • Feature analysis extracts candidate task features with LLMs, merges semantically equivalent labels, and uses correlation tests to retain features associated with strategy performance.The extraction produced 992 raw features; GPT-3.5 merged equivalent features, and Spearman correlation or chi-square tests were used for numerical or categorical features, respectively.Source: §3.2.2 · pp. 6–7

3.3 Evaluation Metrics

  • Pass@1 measures first-attempt task success, Output Token measures generated tokens, and IsGT identifies a strategy that succeeds while using the fewest tokens among successful strategies.The feature analysis also uses P-value for numerical-feature correlation and Cramer’s V for association strength with multiclass features.Source: §3.3 · pp. 7–8

3.4 CEBench Construction

  • CEBench contains 500 real-world tasks curated from SWE-bench, filtered to instances whose ground-truth changes are limited to one file, and labels optimal strategies by evaluating correctness and output-token use.The benchmark has 43 one-step, 184 search-replace, and 95 plan-apply optimal labels; all three strategies failed on the remaining 178 tasks.Source: §3.4 · p. 8

4 Empirical Result

The three editing strategies trade off correctness and token use, with complementary strengths across tasks. Strategy effectiveness is associated with four task features: code size, modification scope, modification type, and task type.

4.1 Results of RQ1

  • 48.2% Pass@1 is achieved by AutoCodeRover (search-replace), while plan-apply uses 941 average tokens—25.8% fewer than search-replace (1,268) and 73.4% fewer than one-step (3,539).The comparison is across all tasks in CEBench.Source: §4.1 · p. 8
  • No strategy solves every task: one-step uniquely solves 17 tasks despite 12.6% Pass@1, and the examples show distinct successes on localized edits and a SparseDataFrame type issue.For the 120-line localized edit, search-replace succeeds with 220 tokens; for the SparseDataFrame issue, only one-step succeeds among the three strategies.Source: §4.1 · p. 9, Figure 3 · p. 9

4.2 Result of RQ2

  • Feature frequency alone does not establish impact: total lines of code (69.46%) and modification type (59.91%) are the most prevalent, but prevalence is distinct from influence on strategy selection.Seven other frequent features appear in over 5% of tasks; most features appear in fewer than 5%.Source: §4.2 · p. 9, Table 2 · p. 10
  • Larger codebases and multi-function edits favor search-replace or plan-apply over one-step, while one-step correlates negatively with both code size (ρ=-0.65) and scope (ρ=-0.60).Plan-apply correlates positively with modification scope (ρ=0.36); search-replace correlates positively with code size (ρ=0.42).Source: §4.2 · p. 10
  • Modification type and task type are associated with strategy effectiveness; the four LLM-clustered task categories were manually checked and cover more than 90% of tasks.The categories are Bug Fix, Feature Add, Refactor, and Performance Optimization.Source: §4.2 · p. 11
  • Four features significantly influence strategy effectiveness: total lines of code, modification scope, modification type, and task type.For numerical features, the study uses Spearman correlations; for categorical features, it uses Chi-square tests and Cramer’s V.Source: §4.2 · pp. 10–11, Table 3 · p. 10

5 Methodology of Ace

Ace uses a fine-tuned model to extract task features, select among three code-editing strategies with explanations, and direct code editing according to the selected strategy. Its workflow separates feature extraction, strategy selection, and code editing.

5 Methodology of Ace

  • Ace has three phases—feature extraction, strategy selection, and code editing—with a fine-tuned model learning feature–strategy mappings from the empirical study.The framework recommends a strategy with an interpretable explanation before code editing.Source: §5 · pp. 11–12, Figure 4 · p. 11

5.1 Fine-tuned CE Strategy Selection Model

  • The selector is fine-tuned from Llama3.1-8B using LoRA on CEBench examples containing the instruction, code, four task features, an optimal strategy, and its rationale.The rationales are generated from feature perspectives, then manually inspected and revised; training uses an 8:2 split for 10 epochs.Source: §5.1 · p. 12

5.2 Feature Extraction

  • For a task, Phase I takes its natural-language instruction, code, and operational definitions of the four high-correlation features, then outputs structured, quantified feature information.The example task is classified as 120 lines, Delete, scope 6, and Bug Fix.Source: §5.2 · pp. 12–13

5.3 Strategy Selection

  • Phase II combines task context with extracted features, recognizes feature patterns and interactions, selects a strategy, and generates a rationale linking features to strategy advantages.In the illustrated case, the selector recommends search-replace for a 120-line file with a local, under-10-line modification to one function.Source: §5.3 · p. 13

5.4 Code Editing

  • Phase III directs an LLM to edit according to the selected strategy, using an adaptive prompt with the instruction, code, strategy, rationale, and strategy-specific protocol.The protocols request a complete rewritten file for one-step, exact search-replace pairs, or a plan followed by placeholder replacement for plan-apply.Source: §5.4 · pp. 13–14
  • Ace returns edited code alongside the selected strategy and rationale, allowing developers to decide whether to accept or refine the edit.The example uses search-replace to alter the targeted code segment while preserving surrounding code.Source: §5.4 · p. 14

6 Evaluation Setup

The evaluation compares Ace with fixed-strategy and adaptive baselines on CEBench-verified, measuring editing performance and strategy-selection quality. It also uses ablations and multiple editing models to examine component contributions and generalization.

6.1 Research Questions; 6.4 Evaluation Metrics

  • Ace is evaluated against fixed-strategy, classifier-based, and instruction-prompting methods on Pass@1, output tokens, Select@1, and rationale accuracy.Select@1 measures correct optimal-strategy selection on the first attempt; two authors judge rationale correctness and use Cohen’s Kappa for agreement.Source: §6.1 · p. 14, §6.4 · pp. 15–16

6.1 Research Questions; 6.2 Evaluation Dataset

  • The study tests whether Ace generalizes by replacing its Phase III editor with three closed-source and three open-source LLMs, comparing against AutoCodeRover.Evaluation focuses on code editing as the repair step, aligned with the benchmark’s editing scope.Source: §6.1 · p. 14

6.2 Evaluation Dataset

  • CEBench-verified contains 650 tasks—429 single-file and 221 multi-file—and is disjoint from Ace’s fine-tuning data; N/A tasks are excluded only from Select@1.The benchmark includes 39 tasks labeled one-step optimal, 152 search-replace optimal, 126 plan-apply optimal, and 333 N/A.Source: §6.2 · p. 15

6.3 Baselines

  • Baselines include CodePlan, AutoCodeRover, and FuseApply as fixed strategies, plus raw-input LLM instruction selection and a CodeBERT feature-based classifier.Ablations remove feature extraction, strategy selection, or individual task features.Source: §6.3 · p. 15

6.5 Experimental Environment

  • Experiments use Claude-Opus-4.1 for Phase III and fine-tuned Llama 3.1-8B for Ace’s Phase I and II selector.The experiments ran on a Linux machine with a 128-core Intel Xeon CPU and 2 TB memory.Source: §6.5 · p. 16

7 Evaluation Result

Ace outperforms fixed and adaptive baselines in editing correctness and token efficiency, while its selector provides accurate strategy choices and rationales. Ablations identify feature extraction and particular task features as important, and results remain favorable across six editing LLMs.

7.1 Overall Effectiveness and Strategy Selection Quality (RQ3)

  • Ace achieves 46.6% overall Pass@1, 21.7% above AutoCodeRover, while reducing output tokens 29.6% versus LLM-IS and 16.4% versus CodeBERT-SC.AutoCodeRover is the strongest fixed-strategy baseline at 38.3% Pass@1; Ace’s overall output-token average is 1,352.Source: §7.1 · pp. 16–17
  • Ace reaches 72.6% Select@1 and 69.9% rationale accuracy, with Cohen’s κ=0.807 inter-rater agreement for rationale assessment.Its Select@1 exceeds LLM-IS (34.1%) and CodeBERT-SC (43.9%); rationale accuracy exceeds LLM-IS (25.9%).Source: §7.1 · p. 17

7.2 Ablation Study (RQ4)

  • Removing feature extraction produces the largest ablation degradation; total lines of code and modification type matter more than modification length or scope.Pass@1 decreases 50.6% without feature extraction; removing total lines of code or modification type decreases it by 27.0% or 24.2%, versus 7.3% and 4.3% for the other features. Removing strategy selection in favor of search-replace decreases Pass@1 by 16.3%, while few-shot prompting instead of the fine-tuned selector decreases it by 24.5%.Source: §7.2 · pp. 17–18

7.3 Generalization Across Different LLMs (RQ5)

  • Across six editing LLMs, Ace averages 21.9% higher Pass@1 and 22.0% fewer output tokens than AutoCodeRover.The reported gains hold across both closed-source and open-source model groups.Source: Table 6 · p. 18, §7.3 · p. 19

8 Discussion

Ace’s locally deployed selector adds modest latency and input overhead, alongside lower editing output-token consumption. The evaluation’s scope and feature data also constrain how broadly its results should be generalized.

8.1 Latency and Token Consumption

  • Ace’s local Llama 3.1-8B selector averages 600 ms per task with no additional API token cost; its 27.3% output-token reduction offsets selection overhead against AutoCodeRover.Phase III guidance adds fewer than 600 input tokens on average.Source: §8.1 · p. 19

8.2 Threats to Internal Validity

  • Evaluation removes repository-level context and measures token efficiency using output tokens only, so it does not fully represent repository-level coding or total token use.The authors state that this setup isolates file-level editing logic and that intermediate reasoning tokens are implementation-specific.Source: §8.2 · p. 19
  • Manual feature quantification and rationale assessment may introduce annotator bias; the authors use independent annotation, consensus, and Cohen’s Kappa to mitigate it.They use Pass@1 as an indirect hallucination-risk indicator and report that Ace does not exacerbate hallucination compared with baselines.Source: §8.2 · p. 20

8.3 Threats to External Validity

  • Feature extraction and evaluation use Python tasks, so performance may decline on other languages; the identified features may also miss requirements of code completion or competitive programming.The authors note that the feature-based methodology remains applicable to other code-editing datasets and languages.Source: §8.3 · p. 20

9 Conclusion

The conclusion presents CEBench’s strategy comparison as evidence that no single code-editing strategy is universally optimal, and introduces Ace as a feature-based adaptive framework. Evaluation on CEBench-verified shows Ace outperforming state-of-the-art baselines.

9 Conclusion

  • CEBench contains 500 real-world editing tasks; its comparison of three strategies identifies four task features associated with strategy effectiveness: code length, modification type, scope, and task type.Ace integrates feature extraction, strategy selection, and code editing; the authors plan to further improve code-editing efficiency.Source: §9 · p. 20

Related papers

Loading 2610.22881v1…