PaperlayerView exact revision on arXiv

Source-linked AI summary

Don't Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction

Hayeong Ryu, Sunhee Jo, Seunguk Yu, YoungBin Kim

arXiv:2609.15559v1cs.CLcs.AI

TL;DR

GEC metrics based on reference or edit overlap can penalize valid rewrites, while judging whether fluent outputs are valid corrections remains challenging. SURE trains a source-conditioned reward evaluator with within-source preferences and criteria- and span-level supervision. On SEEDA, it aligns strongly with human judgments, particularly for fluent rewrite-style corrections, and offers more disentangled criteria diagnostics.

  • Problem

    Reference-based overlap can penalize valid rewrites, and evaluating whether a fluent output is a valid correction of its source remains challenging.

    Source: §1 · pp. 1–2

  • Method

    SURE learns an overall source-conditioned reward from within-source preferences, with criteria-level supervision and span-level grounding for source-side error resolution.

    Source: §1 · p. 2

  • Results

    On SEEDA, SURE aligns strongly with human judgments, especially under +Fluency and rewrite-style comparisons, and provides more disentangled criteria-level diagnostics.

    Source: §1 · p. 2

  • Takeaways & Limitations

    SURE evaluates correction validity through diverse within-source preferences and signals for error resolution, grammaticality, faithfulness, and fluency.

    Source: §1 · p. 2, §2 · p. 3

    SURE uses synthetic, LLM-annotated preference data that may inherit judge-model biases, shows weaker system-level correlations on Wiki, and uses reference-derived ERRANT spans during training.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Grammatical error correction (GEC) evaluation has traditionally relied on reference or edit overlap, which can penalize valid rewrites that differ from gold corrections. Reference-free metrics reduce this dependence, but evaluating whether a fluent output is a valid correction of the source remains challenging. We propose SURE, a source-conditioned reward evaluator trained on within-source preferences spanning minimal-edit and rewrite-oriented corrections. SURE jointly learns an overall reward with criteria-level supervision for grammaticality, faithfulness, and fluency, together with span-level grounding for source-side error resolution. Experiments on SEEDA show that SURE performs competitively against strong baselines, with particular gains on rewrite-style corrections and more disentangled criteria-level diagnostics. Our code is available at https://github.com/hayeonggg/SURE.

1 Introduction

Reference-based overlap can penalize valid rewrites and miss sentence-level meaning and naturalness. SURE uses source-side error-resolution signals and within-source preferences; on SEEDA, it aligns with human judgments, particularly for rewrite-style corrections.

1 Introduction

  • Reference-based edit or surface overlap can penalize valid corrections that differ from gold, under-represent acceptable variation, and favor minimal edits over fluent rewrites.Edit-wise comparison can also miss sentence-level meaning preservation and naturalness, while fluency alone does not establish correction quality.Source: §1 · pp. 1–2
  • Existing reference-free metrics already condition on source and correction, but do not explicitly supervise source-side error resolution or preferences among style-diverse corrections.LLM-based systems often produce holistic rewrites, making it important to assess correction validity rather than reward fluency alone.Source: §1 · pp. 1–2
  • SURE combines span-level source-error resolution supervision with preferences between minimal-edit and rewrite-oriented corrections of the same source.This directly addresses the stated challenge of recognizing diverse valid corrections while ensuring outputs remain faithful corrections of the source.Source: §1 · p. 2
  • SURE aligns strongly with human judgments, especially under +Fluency and rewrite-style comparisons, and provides more disentangled criteria-level diagnostics.Rewrite-oriented data improves rewrite-related comparisons while introducing a trade-off on traditional GEC pairs.Source: §1 · pp. 2–3

2 Problem Formulation

SURE evaluates a candidate as a correction of its source by assigning a source-conditioned reward. The paper characterizes correction validity through grammaticality, faithfulness, and fluency.

2 Problem Formulation

  • SURE assigns a scalar reward Rθ(x, y) to reflect whether candidate y is a valid correction of source x, enabling reference-free evaluation at inference.The scoring target is correction validity rather than overlap with a reference correction.Source: §2 · p. 3
  • Correction validity comprises grammaticality, faithfulness, and fluency: resolving source errors without new ones, preserving meaning and intent, and producing natural, idiomatic English.These criteria characterize validity beyond grammatical correctness alone and allow meaning-preserving rewrites when they improve naturalness.Source: §2 · p. 3

3 SURE: Source-conditioned Unified Reward-based Evaluator

SURE evaluates whether a candidate corrects its source by learning from within-source preferences across minimal edits and rewrites. It combines an overall reward with criteria-level and source-error-span supervision.

3.1 Synthetic Preference Data Generation

  • Candidate pools combine existing references and system outputs with GPT-4o minimal-edit and rewrite-oriented corrections.Sources come from BEA-2019, CoNLL-2014, and JFLEG; CoNLL-derived data is excluded, and sources need at least 10 tokens and two ERRANT-identified errors.Source: §3.1 · p. 3
  • Three LLM judges assess candidate pairs; training retains unanimous overall preferences and aggregates criterion preferences and error-span labels by majority vote.The resulting data contains 2,400 preference pairs from 1,320 source sentences.Source: §3.1 · pp. 3–4
  • References support candidate construction and source-error localization during training, but SURE does not require them at inference.Source–reference pairs are used to identify source-side error spans for span supervision.Source: §3.1 · p. 3

3.2 Reward Modeling

  • SURE predicts an overall correction reward and auxiliary grammaticality, faithfulness, and fluency scores from the source and candidate.The model encodes the concatenated source–candidate input and uses DeBERTa-v3-large fine-tuned with LoRA.Source: §3.2 · p. 4
  • The objective combines pairwise reward ranking with criterion-specific preference losses and binary prediction of whether candidates resolve marked source errors.The criterion losses supervise grammaticality, faithfulness, and fluency; α and β weight criterion-level and span-level supervision, respectively.Source: §3.2 · p. 4
  • Span supervision is used only during training; at evaluation, SURE takes the raw source and candidate without references or span markers.The span head and markers are removed at inference, while the span signals are intended to reduce reliance on surface fluency alone.Source: §3.2 · p. 4

4 Experiments

The experiments evaluate GEC metrics against human judgments on SEEDA, including settings with fluent corrections that differ from reference edits. SURE maintains strong agreement across Base and +Fluency evaluations.

4.1 Experimental Setup

  • SEEDA-E provides edit-level pairwise judgments and SEEDA-S sentence-level judgments across twelve GEC systems, with two fluent human corrections added in +Fluency.The added corrections test whether metrics recognize meaning-preserving, natural rewrites that differ substantially from reference edits.Source: §4.1 · p. 5
  • Sentence-level metric preferences are compared with human judgments using accuracy and Kendall’s τ; system rankings use Pearson’s r and Spearman’s ρ.System-level scores are aggregated over corrections for each system before comparing rankings.Source: §4.1 · p. 5
  • The baseline suite includes reference-based and reference-free GEC metrics, including GPT-4.1-E and GPT-4.1-S instantiated with GPT-4.1-mini.Non-LLM baselines are evaluated using the unified gec-metrics framework.Source: §4.1 · p. 5

4.2 Correction Validity Evaluation

  • Figure 3 reports pairwise accuracy differences between SURE and GPT-4.1-E; positive cells indicate greater human-preference agreement by SURE.GPT-4.1-E is selected as the most directly comparable reference-free LLM evaluator because both assess edit appropriateness conditioned on source and candidate.Source: Figure 3 · p. 6
  • SURE maintains strong agreement with human judgments across Base and +Fluency settings at both system and sentence levels.Reference-based metrics often lose agreement under +Fluency, while reference-free metrics are generally more stable.Source: §4.2 · pp. 5–6

5 Analysis

The analyses find that SURE aligns well with human preferences, including on rewrite-style comparisons, while rewrite-oriented training data produces gains and trade-offs across pair types. Human checks support the preference labels, and SURE’s criteria scores separate correction profiles.

5.1 Evaluation Behavior of SURE

  • SURE achieves the highest pairwise accuracy across system-style groups on both SEEDA subsets, with its advantage clearest in rewrite-related comparisons.Against GPT-4.1-E, positive system-pair accuracy differences appear across many pairs, especially around rewrite-style systems such as GPT-3.5 and REF-F.Source: §5.1 · pp. 6–7
  • SURE is not highly sensitive to auxiliary criteria-level and span-level loss weights, with stable system- and sentence-level performance; auxiliary supervision generally improves or preserves performance over pairwise-only training (0.0, 0.0).Source: §5.1 · p. 7

5.2 Effect and Reliability of Preference Data

  • Adding rewrite-oriented preference data improves system-level correlation on both SEEDA subsets, especially under +Fluency, while sentence-level accuracy is largely preserved.The reported small SEEDA-E decrease reflects a mild trade-off with edit-level judgments favoring local corrections; pair-type analysis finds gains on rewrite-related pairs but drops on Trad. only pairs.Source: §5.2 · p. 7
  • Unanimous LLM agreement yields higher overall accuracy and κ than individual judges or majority voting.The authors use unanimous consensus as the filtering criterion for training pairs.Source: §5.2 · p. 7

5.3 Criteria-Level Diagnostics

  • SURE’s faithfulness and fluency scores are nearly independent, unlike SOME’s strongly negative correlation; grammaticality and faithfulness also separate, while grammaticality and fluency remain closely related.On TMU-GFM, SURE shows reasonable zero-shot transfer on confident grammaticality and fluency pairs, while meaning preservation remains challenging.Source: §5.3 · p. 8

5.4 Human Validation of Preference Data

  • Retained unanimous pairs have 62.0% human consensus versus 38.0% for discarded pairs, and retained labels match human overall preferences in 77.4% of consensus cases.The consensus-rate difference is statistically significant (p = 0.027); discarded-pair label agreement is 57.9%.Source: §5.4 · p. 8
  • For retained pairs with human consensus, LLM–human agreement is 92.3% for grammaticality, 84.0% for faithfulness, and 75.9% for fluency.Together with the 77.4% overall agreement, the authors report broad consistency with independent human judgments.Source: §5.4 · p. 8
  • Human annotators favor rewrites more often than LLM judges on consensus rewrite–non-rewrite pairs, yet agree with seven of eight LLM rewrite preferences.The rewrite preference rates are 83.3% for humans and 66.7% for LLM judges.Source: §5.4 · p. 8

6 Related Work

Prior GEC evaluation work moves from reference overlap toward source-conditioned quality judgments, but gaps remain in supervising source-side error resolution and preferences across correction styles. SURE combines those signals with preference-based evaluation.

6.1 Reference-Based Metrics for GEC

  • Reference-based refinements improve interpretability and validity, but remain tied to gold corrections or reference-derived edits.CLEME 2.0 decomposes edit outcomes into hit, wrong, under-, and over-correction (Ye et al., 2025); JELV adds an edit-level validity judge (Zhan et al., 2026).Source: §6.1 · p. 9

6.2 Reference-Free and LLM-Based GEC Evaluation

  • Reference-free metrics condition on source and correction, but prior approaches do not combine source-side error-resolution supervision with within-source preferences across styles.Recent LLM-based evaluators use edit-level and sentence-level prompts, while SURE focuses on this combined supervision design.Source: §6.2 · p. 9

6.3 Preference and Reward Modeling for Evaluation

  • Pairwise preference modeling trains evaluators from judgments, while LLM-judge judgments can vary with prompts, judge models, and evaluation criteria.SURE evaluates correction quality through grammaticality, faithfulness, fluency, and span-level error resolution.Source: §6.3 · p. 9

7 Conclusion

SURE learns correction validity from style-diverse preferences with criteria-level and span-level supervision. On SEEDA, it aligns strongly with human judgments, particularly for fluent rewrite-style corrections, and provides diagnostics for grammaticality, faithfulness, and fluency.

7 Conclusion

  • SURE aligns strongly with human judgments on SEEDA, particularly for fluent rewrite-style corrections.Source: §7 · p. 9
  • SURE learns correction validity from style-diverse preferences using criteria-level and span-level supervision.Source: §7 · p. 9
  • SURE provides useful diagnostics for grammaticality, faithfulness, and fluency.Source: §7 · p. 9

Limitations

SURE's limitations include potential biases in its training preferences, uneven domain generalization, and reference-derived span supervision during training.

Limitations

  • Synthetic and LLM-annotated preference data may transmit biases from the judge models used to construct it.Source: Limitations · p. 10
  • System-level correlations are weaker on Wiki despite competitive transfer on FCE and at sentence level, suggesting calibration may remain sensitive to domain shift.Source: Limitations · p. 10
  • Although inference requires no references, training uses reference-derived ERRANT spans for span-level supervision.Source: Limitations · p. 10

A Dataset Statistics

The training dataset uses unanimous LLM-judge preferences over correction pairs and includes comparisons between minimal-edit and rewrite-oriented outputs. It contains 2,400 pairs from 1,320 sources, with substantial GPT-4o coverage.

A Dataset Statistics

  • The final dataset contains 2,400 preference pairs from 1,320 BEA-2019 and JFLEG source sentences; CoNLL-2014-derived data were excluded to avoid SEEDA source overlap.Source: A Dataset Statistics · p. 13, Table 9 · p. 13
  • Of 8,093 candidate pairs judged, 4,846 (59.9%) received unanimous overall preferences and 3,247 received split judgments; 2,400 unanimous pairs were sampled.Source: A Dataset Statistics · p. 13
  • 69.6% versus 53.8%: rewrite-oriented GPT-4o corrections had a higher win rate than its minimal-edit variant.Source: Table 10 · p. 13, A Dataset Statistics · p. 13
  • GPT-4o-generated corrections appeared in 87.0% of pairs, accounting for 62.5% of preferred candidates and 40.4% of dispreferred candidates.Source: Table 10 · p. 13, A Dataset Statistics · p. 13
  • Preference pairs include comparisons between rewrite-style and minimal-edit corrections, as well as fluency-oriented and rewrite-style corrections.Table 11 groups pairs by preferred and dispreferred correction styles.Source: Table 11 · p. 13, A Dataset Statistics · p. 13
  • Each training instance includes criteria-level preferences, span-resolution labels for both candidates, and source, candidate, and metadata fields.Criteria labels indicate whether the overall preferred or dispreferred correction wins under each criterion.Source: Table 12 · p. 13, A Dataset Statistics · p. 13

B System-Ranking Analysis under +Fluency

Under +Fluency, SOME and SURE recover the human top-two system ordering that ERRANT misses. Transfer is reasonable at sentence level, while Wiki system-level calibration and faithfulness remain more difficult.

B System-Ranking Analysis under +Fluency

C.1 Cross-Domain Transfer on GMEG

  • SURE shows competitive performance on FCE and stronger sentence-level results on Wiki, but Wiki system-level correlations remain below SOME and IMPARA.The evaluation is zero-shot, without additional fine-tuning; the results indicate reasonable sentence-level transfer but domain-sensitive system-level calibration.Source: §C.1 · p. 14, Table 14 · p. 14

C.2 Criteria-Level Transfer on TMU-GFM

  • On confident TMU-GFM pairs, SURE achieves reasonable zero-shot accuracy on grammaticality and fluency, while faithfulness remains its weakest axis.SOME performs better across all axes on all pairs, consistent with its in-domain axis-level supervision; the faithfulness gap reflects difficulty aligning SURE’s criterion with TMU-GFM meaning-preservation annotations.Source: §C.2 · p. 14, Table 15 · p. 14

C.3 Additional Criteria-Pair Diagnostics

The diagnostics show that SURE separates faithfulness from grammaticality more than SOME does, despite both metrics’ close grammaticality–fluency relationship. The appendices also specify distinct generation, evaluation, and annotation prompts.

C.3 Additional Criteria-Pair Diagnostics

  • Both metrics show near-perfect grammaticality–fluency correlation, but SURE keeps grammaticality and faithfulness much less correlated than SOME, yielding a more separated criteria space.SOME instead shows a strong negative grammaticality–faithfulness correlation; the figures report the additional criteria-pair correlations.Source: §C.3 · pp. 14–15, Figure 7 · p. 15, Figure 6 · p. 15

D.1 Correction Generation Prompts

  • The minimal-edit prompt requests a correction, while the rewrite-oriented prompt permits restructuring and lexical changes if meaning is preserved and the surface form differs.Both prompts ask for only the corrected or rewritten sentence, without commentary.Source: §D.1 · p. 15

D.2 GPT-4.1 Evaluation Prompts

  • The evaluation prompts contrast ranking overall sentence quality with ranking edits, with the edit-focused prompt penalizing meaning changes and unnecessary edits.The overall-quality prompt asks evaluators to consider grammaticality, fluency, naturalness, clarity, and source meaning.Source: §D.2 · pp. 15–16

D.3 Pairwise GEC Annotation Prompt

  • Pairwise annotators judge grammaticality, faithfulness, fluency, and overall preference separately, treating minimal edits and rewrites as equally valid.The prompt also asks annotators to assess whether listed source errors were resolved, missed, or only partially addressed.Source: §D.3 · p. 17

Related papers

Loading 2609.15559v1…