Grammatical Error Correction: A Survey of the State of the Art

1. Introduction

• The field moved from parser-applied hand-coded mal-rules to data-driven methods and systems that treat correction as translation.The survey focuses especially on recent deep-learning approaches, task characteristics, evaluation, and multilingual GEC challenges.Source: §1 · p. 2

1.1 The Task

• GEC boundaries are unclear, spanning orthographic, morphological, discourse-level, and context-dependent cases that can be difficult to classify as grammatical errors.The same correction may depend on communicative intent, dialect, or genre.Source: §1.1 · pp. 2–4
• Corpus practice defines GEC broadly: shared-task annotations generally favor minimal meaning-preserving edits, while some learner corpora contain extensive rewrites.As a result, the label “Grammatical” Error Correction also covers spelling and discourse errors; “Language Error Correction” would be more descriptive.Source: §1.1 · p. 3
• Error annotation spans replacement, omission, and insertion, with complex or multiple errors requiring sequences of edits and schemes varying in granularity.ERRANT standardizes parallel sentence pairs into 25 main error categories, supporting consistent training, evaluation, and error-type analysis.Source: §1.1 · pp. 3–4
• Sentence-level correction can miss context-dependent errors, and corpora with few alternative corrections can underestimate system performance.Document-level correction could in principle address some issues, but its development is hampered by the lack of appropriately structured corpora.Source: §1.1 · p. 4

1.2 Survey Structure

• The survey organizes its coverage around core approaches, supplementary techniques, and data augmentation, aiming to complement rather than duplicate earlier surveys.The authors note that some overlap with Wang et al. (2021) is unavoidable for established techniques.Source: §1.2 · p. 4

2. Data

• Modern GEC systems depend on large, high-quality datasets, but collecting annotated data is laborious and resources are scarcer than in machine translation.The section reviews human-annotated corpora and leaves artificial data for a later section.Source: §2 · p. 5

2.1 Annotation Challenges

• Annotation must balance minimal grammatical edits against fluent rewrites, while guidelines should clarify editing extent because the boundary is difficult to define.Fluent corrections are described as more desirable, but minimal corrections are generally easier for humans and machines.Source: §2.1 · pp. 5–6
• Subjective correction choices and edit-span conventions affect analysis and edit-based evaluation, making consistent annotation guidelines important.Explicit error-type frameworks can add cognitive burden and may influence which corrections annotators choose.Source: §2.1 · pp. 5–6
• Converting character-level edits to token-level edits and handling sentence-boundary changes complicate preprocessing; cross-sentence edits are ignored by sentence-based systems.Paragraph- or document-based systems are unaffected by this sentence-based limitation.Source: §2.1 · pp. 6–7

2.2.1 Benchmark English Datasets

• FCE and NUCLE are prominent learner benchmarks, with single-annotator corrections and distinct error-type schemes; CoNLL-2014 remains a common test set.FCE covers intermediate learners and 88 error types; NUCLE has 28, and CoNLL-2014 has multiple reference sets but limited topic and learner diversity.Source: §2.2.1 · pp. 7–8
• Lang-8 offers large-scale, unconstrained learner text but noisy user corrections, while JFLEG emphasizes fluent edits and W&I+LOCNESS supplies balanced ability levels.JFLEG has four references per sentence; W&I+LOCNESS test references are evaluated through the BEA-2019 competition platform.Source: §2.2.1 · pp. 8–9

2.2.2 Other English Datasets

• Other English corpora broaden domain and scale, but AESW is domain-specific and GHTC is dominated by software-documentation spelling and orthographic edits.EFCamDat annotations may be incomplete because teachers produced them for student feedback rather than system development.Source: §2.2.2 · pp. 9–11

2.3 Non-English Datasets

• Resources described for German, Japanese, Russian, Ukrainian, Arabic, Chinese, and Czech support research beyond English, with varied annotation schemes and corpus scopes.MuCGEC provides multiple references across Chinese sources, while GECCC combines learner, native-student, and informal texts in a large Czech corpus.Source: §2.3 · pp. 11–13

3.1 Classifiers

• Classifiers suit common ESL article and preposition errors with small candidate sets, but handle specific error types and local contexts poorly.Combining classifiers can also produce inconsistent predictions because classifier order matters.Source: §3.1 · pp. 14–15

3.2 Statistical Machine Translation

• SMT treats correction as reconstructing a correct sentence from an erroneous one, combining language and translation models; limited, quality-sensitive parallel data remains a key constraint.Its success on general correction helped make SMT dominant, and artificial errors are a common way to address scarce training data.Source: §3.2 · pp. 15–17

3.3 Neural Machine Translation

• NMT models correction end to end with an encoder–decoder network and has achieved state-of-the-art performance across GEC tasks.Unlike SMT, it does not require separately trained and tuned components or complex GEC-specific feature engineering.Source: §3.3 · p. 17

3.3.1 Recurrent Neural Networks; 3.3.2 Convolutional Neural Networks

• RNN-based GEC uses attention-based sequence models, while CNNs capture local context through convolution and wider context through stacked layers.The 7-layer CNN model of Chollampatt and Ng (2018a) was the first NMT-based model reported to significantly outperform prior SMT-based models.Source: §3.3.1 · p. 17, §3.3.2 · p. 18

3.3.3 Transformers

• Transformers rely on self-attention for input representations and parallelize better across GPUs than RNNs; synthetic-data pre-training and direct fine-tuning of large pre-trained models have both supported strong GEC systems.Transformer-based systems also use copy mechanisms and additional sentence or document context.Source: §3.3.3 · p. 18

3.4 Edit-based approaches

• Edit-based methods generate operations rather than whole corrected sentences, typically speeding inference and improving transparency, but token-based edits can miss complex multi-token changes.Sequence tagging trades model size against coverage for token-dependent edits, while sequence-to-sequence edit generation is more flexible but slower than tagging.Source: §3.4 · pp. 19–20

3.5 Language Models for Low-Resource and Unsupervised GEC; 3.5.1 Language models as Discriminators

• Language-model approaches can use monolingual data, including for low-resource or unsupervised GEC; traditional methods rank candidate corrections by language-model probability, and BIFI iteratively filters synthetic training pairs.BIFI trains a breaker and fixer through feedback rounds, using an LM critic to filter sentence pairs.Source: §3.5 · p. 20, §3.5.1 · pp. 20–21

3.5.2 Language models as Generators

• Prompted large language models can generate corrections without explicit GEC training, but standard-test benchmarking remained limited and LM probability does not guarantee grammaticality.The survey reports that small-sample studies found a tendency to overcorrect fluency on datasets designed for minimal corrections.Source: §3.5.2 · p. 21

4.1 Re-ranking

• Reranking selects among an MT system’s n-best corrections using additional features such as fluency, edit similarity, length, syntax, and error detection.The features can be applied after decoding without adapting fine-grained model smoothing.Source: §4.1 · p. 22

4.2 Ensembling and System Combination

• System combination can integrate substantially different GEC systems’ strengths, while ensembles typically average predictions from similarly configured models.Combination models can operate as black boxes, using system outputs without access to internals or prediction probabilities.Source: §4.2 · p. 23

4.3 Multi-task learning

• Multi-task learning shares representations across related objectives, including error labels, error types, learner proficiency, and sentence- or token-level error detection.Labels for auxiliary error detection can be extracted automatically from existing datasets using alignment tools such as ERRANT.Source: §4.3 · p. 23

4.4 Custom inference methods

• Iterative decoding gives a model another chance to correct missed errors, while span-based and aggressive decoding target faster inference.Span-based correction reduced inference time by almost 50%; aggressive decoding yielded an almost ten time speedup, as reported in the survey.Source: §4.4 · p. 24

4.5 Contextual GEC

• Sentence-level systems miss cross-sentence dependencies; Transformer studies found local context of up to two sentences useful, while longer context was not necessary for improved performance.Because references often do not mark context-dependent edits, researchers proposed synthetic cross-sentence tense errors and document-level evaluation.Source: §4.5 · pp. 24–25

4.6 Generative Adversarial Networks

• Adversarial training has improved both sequence-to-sequence correction and sequence-labelling GEC over non-adversarial baselines.The surveyed approaches train a generator against a discriminator distinguishing generated output or edits from human references or edits.Source: §4.6 · p. 25

5. Data Augmentation

• High-quality parallel corpora contain only roughly 50k sentence pairs, while larger resources such as Lang-8 are noisy; augmentation is now a staple of recent GEC models.The quality of synthetic data remains difficult to evaluate directly.Source: §5 · p. 25

5.1 Synthetic Data Generation

• Synthetic parallel data pairs a corrupted sentence from clean monolingual text with its original as the reference; common generation strategies are noise injection and back-translation.Human annotation is expensive and difficult, increasing interest in synthetic corpora as neural models require more data.Source: §5.1 · p. 26

5.1.1 Noise Injection

• Noise injection perturbs clean text through rule-based character- or word-level edits, corpus-derived error patterns, or inverse correction patterns applied to grammatical sentences.Rules may be applied dynamically during training; pattern-based methods can use lexical or POS-level corrections.Source: §5.1.1 · pp. 26–27

5.1.2 Back-translation

• Back-translation trains a noisy-channel model on reversed GEC pairs; Kiyono et al. (2019) report better performance than rule-based injection, while interpolating generation systems tends to improve synthetic data.Stahlberg and Kumar (2021) report that Seq2Edit-generated edit sequences outperform direct corruption, and sampling outperforms beam search in their experiments.Source: §5.1.2 · p. 28

5.1.3 Round-trip Translation

• Round-trip translation uses an intermediate language to generate noisy sentence pairs, assuming translation errors make the returned text noisy relative to the input.A related approach treats lower-quality translation output as the noisy sentence and higher-quality output as the reference.Source: §5.1.3 · p. 28

5.2 Augmenting Official Datasets

• Dataset augmentation reduces noise by correcting or down-weighting unreliable examples, while model enhancement targets training–inference mismatches and model weaknesses.Methods include delta-log perplexity reweighting, GAN-generated errors, multi-round edit-type training, contrastive learning, and reinforcement-learning-based sampling.Source: §5.2 · pp. 28–29

6.1 MaxMatch

• M2 compares hypothesis edits with annotated reference edits, counts true positives, false positives, and false negatives, and calculates an Fβ-score with β = 0.5 commonly used to weight precision twice as much as recall.The survey introduces M2, ERRANT, and GLEU among the commonly used GEC metrics.Source: §6.1 · pp. 29–30
• Edit overlap can miss a valid correction when the hypothesis and reference divide the same change into different edits.For example, [has eating →was eaten] can be represented as one edit or as [has →was] and [eating →eaten].Source: §6.1 · p. 30
• M2 uses Levenshtein alignment to explore edit combinations that maximize reference matching and selects the best-performing reference edit set when multiple sets exist.This addresses a limitation of the earlier HOO shared-task scorer, which could return erroneous scores.Source: §6.1 · p. 30

6.2 ERRANT

• ERRANT uses linguistically enhanced Damerau–Levenshtein alignment and rule-based error classification to calculate F-scores by error type as well as overall.This supports detailed analyses of different systems’ strengths across error types.Source: §6.2 · p. 30
• ERRANT provides three error-type granularities and can analyze both error detection and correction; it currently supports English, with independent extensions for several other languages.Its labels range from three edit operations to 25 main types and 55 full types.Source: §6.2 · pp. 30–31

6.3 GLEU

• GLEU rewards hypothesis n-grams shared with a reference but not the original, and penalizes those shared with the original but not the reference.It was motivated by the arbitrariness and collection cost of human-annotated edit spans.Source: §6.3 · p. 31
• GLEU computes precision over n-grams of lengths 1–4 by default and includes a brevity penalty for hypotheses shorter than references.The metric is inspired by BLEU but uses original text as well as corrected references.Source: §6.3 · p. 31
• When multiple references are available, GLEU iteratively samples one and averages the score over 500 iterations.Source: §6.3 · p. 32

6.4.1 Reference-based Metrics

• The I-measure scores relative textual improvement against a no-change baseline, while GMEG ensembles metric features with ridge regression to predict human scores.I-measure values range from -1 to 1; GMEG was designed for more consistent human-judgment correlation across domains.Source: §6.4.1 · p. 32
• GoToScorer weights edits by how many systems correct them, but its difficulty weights depend on the participating systems; SErCl uses syntax-based labels, while SERRANT combines aspects of SErCl and ERRANT.SErCl can produce uninformative labels and is not meaningful for some errors, such as spelling.Source: §6.4.1 · pp. 32–33
• PT-M2 combines edit-based and pretrained semantic metrics, measuring pretrained similarity at the edit level after off-the-shelf metrics correlated poorly with sentence-level human judgments.It achieved the highest reported human-judgment correlation on the CoNLL-2014 test set to date, but the survey urges caution about correlation as an evaluation criterion.Source: §6.4.1 · p. 33

6.4.2 Reference-less Metrics

• Reference-less GBMs use a benchmark GEC system or a grammaticality predictor, but require an existing system and do not detect meaning changes; GFM adds fluency and meaning-preservation scores.GFM’s meaning-preservation component measures shallow similarity through overlapping content words.Source: §6.4.2 · p. 33
• USim compares semantic graphs to assess meaning preservation and is intended to complement other metrics; SOME fine-tunes BERT to predict human scores for grammaticality, fluency, and meaning.SOME correlates more strongly with human judgments than GFM, though both share limitations.Source: §6.4.2 · pp. 33–34
• The Scribendi Score combines language-model perplexity with token and Levenshtein-distance ratios, but can reward false positives when plausible alternatives are similarly scored.The survey cautions that the score is not a robust way to evaluate a GEC system.Source: §6.4.2 · p. 34
• IMPARA trains edit-quality and semantic-similarity models on parallel data, then operates without references; it is domain-sensitive and its scores are not currently human-interpretable.It shows comparable or better human-judgment correlation than SOME and requires neither system outputs nor human judgments for training.Source: §6.4.2 · p. 34

Metric Reliability

• Human pairwise rankings of sampled system outputs became influential evidence used to justify GLEU over M2 and motivate most reference-less metrics.The evaluation framework ranked randomly selected groups of five outputs, allowing ties.Source: §6.5 · pp. 34–35
• Correlation coefficients differed substantially between two studies that essentially repeated the same experiment on different samples, casting doubt on conclusions drawn from this methodology.The survey notes that the methodology was later found problematic.Source: §6.5 · p. 35
• Chollampatt and Ng (2018b) found no evidence that GLEU outperformed M2 for ranking systems.Source: §6.5 · p. 35
• Grammaticality ratings are highly subjective and often show very low inter-annotator agreement, including when weighing one major error against two minor errors.This subjectivity is offered as a reason metric-correlation experiments proved unreliable.Source: §6.5 · p. 35
• IAA was 0.9-0.99 Pearson/Spearman at the corpus level but 0.3-0.6 Pearson/Spearman at the sentence level in Napoles, Nădejde, and Tetreault (2019).Their results also indicate that dataset choice affects metric performance, likely because humans judge different error-type distributions inconsistently.Source: §6.5 · p. 35
• The I-measure's reported relationship with human judgments shifted from weak negative correlation to good sentence-level correlation and later best-singular-metric status across domains.Reliable ways to evaluate automatic metrics remain an active research area.Source: §6.5 · pp. 35–36

6.6 Evaluation Best Practices

• CoNLL-2014 is conventionally scored with M2, JFLEG with GLEU, and BEA-2019 with ERRANT; M2 F0.5 is not equivalent to ERRANT F0.5.These pairings persist partly for historical reasons and support fair comparisons with prior work.Source: §6.6 · p. 36

6.6.1 Caveats

• Using different metrics for different test sets may be undesirable, so multi-corpus evaluation and a metric designed to reduce dataset bias can provide greater confidence in versatility.Mita et al. (2019) recommend multiple corpora, while Napoles, Nădejde, and Tetreault (2019) recommend their trained metric.Source: §6.6.1 · p. 36

6.6.2 Recommendations

• Evaluate BEA-2019 with ERRANT and CoNLL-2014 with M2 to compare against established benchmarks and assess performance across learner backgrounds and ability levels.BEA-2019 is described as diverse; CoNLL-2014 is a widely used indicator of system performance.Source: §6.6.2 · p. 37
• Evaluate on GMEG or CWEB with ERRANT to include native-speaker errors beyond the learner-focused BEA-2019 and CoNLL-2014 sets.This can give a more complete picture of system generalisability.Source: §6.6.2 · p. 37
• Use JFLEG with GLEU to test more complex fluency edits, while noting that the set is small and not all its edits are fluency edits.Researchers have seldom reported JFLEG GLEU in recent years (Gong et al. 2022).Source: §6.6.2 · p. 37
• A score’s interpretation depends on in-domain training or fine-tuning, metric optimization, and whether evaluation uses only a specific target test set.Direct score comparisons can reward specialized systems over lower-scoring but more versatile ones.Source: §6.6.2 · p. 37

7.1 System Description

• Recent systems build on Omelianchuk et al. (2020), Sun et al. (2021), and Kiyono et al. (2019), whose approaches combine sequence tagging or modified BART and Transformer-big models with synthetic training data.The cited systems used 9 million, 300 million, and 70 million synthetic sentence pairs, respectively.Source: §7.1 · pp. 37–38
• Successors to Omelianchuk et al. (2020) use larger pretrained models, edit scoring or majority voting, GANs or multi-turn training for edit interdependence, and iterative synthetic-data refinement.These extensions are attributed to Sorokin (2022), Tarnavskyi, Chernodub, and Omelianchuk (2022), Parnow, Li, and Zhao (2021), Lai et al. (2022), and Yasunaga, Leskovec, and Liang (2021).Source: §7.1 · p. 38
• Other extensions add a precision-recall trade-off hyperparameter to Sun et al. (2021), incorporate BERT into Kiyono et al. (2019), or use self-refinement data augmentation.These approaches are attributed to Sun and Wang (2022), Kaneko et al. (2020), and Mita et al. (2020).Source: §7.1 · p. 38

ERRANT

• Table 8 uses abbreviations for corpora and techniques, including ensemble, multi-turn decoding, precision-recall trade-off, re-ranking, system combination, and voting combination.Its corpus codes include NUCLE, FCE, Lang-8, W&I, cLang-8, and Cambridge Learner Corpus.Source: ERRANT · p. 39, Table 8 · p. 39

7.2 Analysis

• Differences in training data, pretrained models, and performance-boosting techniques make Table 8 systems rarely directly comparable.The authors therefore infer relative advantages only from the wider context.Source: §7.2 · p. 40
• More than half a billion synthetic sentences did not prevent Stahlberg and Kumar (2021) from being outperformed by systems using orders of magnitude less data.The survey reports no clear relationship between artificial-data quantity and performance; some systems remain competitive without artificial data.Source: §7.2 · p. 40
• The top three latest state-of-the-art systems use at least two pretrained models, and the survey generally associates multiple architectures with improved performance.The authors suggest different pre-training tasks capture complementary aspects of language, while noting a possible model-complexity and runtime trade-off.Source: §7.2 · p. 40
• Systems using the most performance-boosting techniques typically score highest, while ensembling or system combination adds model complexity and runtime cost.These methods can use the strengths of several systems and mitigate neural-model instability.Source: §7.2 · p. 41

Domain Generalisation and Personalised Systems

• GEC performance may not transfer across domains or learner profiles, motivating broader-domain corpora and tools adaptable to users’ ability levels and first languages.Systems performing well in one domain do not necessarily perform well in another.Source: §8 · p. 41

Feedback Comment Generation and Model Interpretability

• Educational GEC needs explanations of why errors were corrected, while interpretable output and estimates of prediction quality remain active research needs.Example-based systems can provide similar corrections in different contexts to add credibility to a prediction.Source: §8 · p. 41

Semantic Errors

• Semantic errors such as collocations, idioms, multi-word expressions, and fluency edits remain an area for improvement despite work on some of these phenomena.The survey cautions against neglecting content-word errors in favor of frequent function-word errors.Source: §8 · pp. 41–42

Contextual GEC

• Sentence-level systems handle cross-sentence or document-level errors poorly, while most current datasets do not facilitate multi-sentence evaluation.The survey calls for paragraph- or document-level datasets to encourage contextual GEC.Source: §8 · p. 42

System Combination

• Systems built with different approaches can have complementary strengths, and combining them can significantly improve performance.The survey identifies understanding when and how to combine approaches as a promising research area.Source: §8 · p. 42

Training Data Selection and Unsupervised Approaches

• Training-data selection is a challenge because massive synthetic pretraining is computationally expensive and environmentally unfriendly, while the need for so much data remains questionable.The survey also notes that parallel corpora require laborious annotation and that high-quality synthetic errors are difficult to generate reliably.Source: §8 · p. 42

Multilingual GEC and Spoken GEC

• A single multilingual GEC system is desirable instead of separate systems for each language, while spoken GEC adds accent recognition, disfluency detection, and utterance segmentation challenges.Progress in spoken GEC has been hindered by limited available data.Source: §8 · p. 43

Improved Evaluation

• Current evaluation does not properly account for alternative corrections or untokenized text, despite those being relevant to end-user preferences.Most metrics require tokenized text, while systems typically return one best correction rather than a short n-best list.Source: §8 · p. 43

9. Conclusion

• The survey examines how communicative intent complicates defining errors, reviews English and multilingual corpora, and notes growing work on non-English GEC.The languages mentioned include Arabic, Chinese, Czech, German, and Russian.Source: §9 · p. 43
• It traces GEC methods from error-type-specific classifiers to neural machine translation and edit-based sequence labeling, alongside supplementary techniques and artificial-data methods.The survey also highlights low-resource systems as potentially less resource intensive and more readily extended to other languages.Source: §9 · p. 43
• Robust GEC evaluation remains unsolved, with the survey reviewing metric strengths and weaknesses and the difficulty of correlating metric scores with subjective human judgments.It covers both reference-based and reference-less metric approaches.Source: §9 · p. 44
• The survey analyzes recent progress, including techniques and resources in relation to model efficiency, and outlines future work.It is intended as a resource for newcomers and readers tracking recent developments.Source: §9 · p. 44

• A Boyd 2018 entry is titled “Using Wikipedia Edits in Low Resource Grammatical Error Correction.”Source: 2013. Memory-based Grammatical Error · pp. 44–45
• The MERLIN corpus entry describes it as learner language and the CEFR.Source: 2013. Memory-based Grammatical Error · p. 45
• Brockett, Dolan, and Gamon’s 2006 entry is titled “Correcting ESL Errors Using Phrasal SMT Techniques.”Source: 2013. Memory-based Grammatical Error · p. 45
• Brown et al.’s 2020 entry is titled “Language Models are Few-Shot Learners.”Source: 2013. Memory-based Grammatical Error · p. 45
• The references also include Bryant’s 2019 thesis on error-type annotation and a work by Bryant and Briscoe on language-model-based correction without annotated training data.Source: 2013. Memory-based Grammatical Error · p. 45