PaperlayerView exact revision on arXiv

Source-linked AI summary

VAA-CSEC: Vote-guided Advantage Allocation for Chinese Semantic Error Correction

Yitong Han, Nankai Lin, Juan Luo, Hongyan Wu, Lianxi Wang, Shengyi Jiang

arXiv:2609.36804v1cs.CLcs.AI

TL;DR

Chinese semantic error correction remains underexplored, and LLMs face over-correction as well as an unclear relationship between CoT and self-consistency decoding. VAA-CSEC combines CoT, task-specific rewards, and vote-guided RL; it reports leading results on CSED-C and NaSGEC-Exam.

  • Problem

    Chinese semantic errors are subtle and complex, leaving CSEC underexplored; LLM correction also faces over-correction and an unclear interaction between CoT reasoning and self-consistency decoding.

    Source: §1 · pp. 1–2

  • Method

    VAA-CSEC combines CoT distillation, supervised fine-tuning, and reinforcement learning, using a minimal-editing reward and GLPO to align training with vote-based self-consistency inference.

    Source: §1 · p. 2

  • Results

    VAA-CSEC achieves 47.72% F0.5 on CSED-C, the highest recall among all methods at 42.15%, and 41.55% F0.5 on NaSGEC-Exam, a new state of the art.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors report that CoT's benefit in CSEC depends on sampling strategy: it can hurt with greedy decoding, while self-consistency yields steadily growing gains.

    Source: §5 · p. 8

    Experiments focus exclusively on CSEC, evaluated on CSED-C and NaSGEC-Exam; the rollout-size parameter study covers only N ∈{4, 6, 8}.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Chinese Semantic Error Correction (CSEC) targets semantic errors in Chinese text, which are typically more subtle and complex than spelling and grammatical errors but remain relatively underexplored. Existing LLM-based approaches face two recurring obstacles in this task: over-correction, and unclear interaction between Chain-of-Thought (CoT) reasoning and self-consistency decoding, such that the benefits brought by CoT cannot be reliably transferred to final corrections. We propose Vote-guided Advantage Allocation for CSEC (VAA-CSEC), a multi-stage framework that combines CoT distillation, Supervised Fine-Tuning (SFT), Reinforcement Learning (RL) and self-consistency decoding. During RL, we design a task-specific reward function that directly aligned with the minimal-editing principle of CSEC. We further introduce Group-Level Relative Policy Optimization (GLPO), which reallocates GRPO advantages according to the margin between individual rollout rewards and the vote-aggregated group reward, aligning the RL training objective with the self-consistency objective used at inference time. Experiments on CSED-C and NaSGEC-Exam show that VAA-CSEC outperforms all LLM-based baselines on CSED-C with an F0.5 of 47.72%, achieves the highest recall of 42.15% among all methods, and establishes a new state of the art of 41.55% F0.5 on NaSGEC-Exam.

1 Introduction

CSEC remains underexplored despite the subtlety of semantic errors and their potential to cause ambiguity in downstream applications. VAA-CSEC combines CoT, task-specific reward design, and vote-guided RL, with strong results on two benchmarks.

1 Introduction

  • CSEC remains underexplored because semantic errors are subtle and difficult to correct, despite their potential to cause ambiguity and misinterpretation downstream.Source: §1 · p. 1
  • LLM correction risks over-editing, while CoT's effect depends on decoding: it underperforms with greedy decoding but improves with self-consistency.Source: §1 · pp. 1–2
  • VAA-CSEC combines CoT distillation, supervised fine-tuning, reinforcement learning, and self-consistency decoding to address CSEC's correction and decoding challenges.Source: §1 · p. 2
  • Its task-specific reward constrains output format, rewards effective edits, and penalizes over-correction, while GLPO reallocates GRPO advantages using the margin between rollout rewards and a vote-aggregated group reward.Source: §1 · p. 2
  • VAA-CSEC achieves the best performance among LLM-based methods on CSED-C with 47.72% F0.5 and the highest recall among all methods at 42.15%, while establishing a 41.55% F0.5 state of the art on NaSGEC-Exam.Source: §1 · p. 2

2 Related Work

Related work traces Chinese text correction from rule-based and statistical methods through neural and LLM approaches, while noting persistent over-correction and underexplored semantic correction. RL-based correction methods increasingly use tailored rewards and consensus mechanisms.

2.1 Traditional Chinese Text Error Correction Methods

  • Chinese text correction progressed from rule-based and statistical methods to Seq2Seq and Seq2Edit neural paradigms; semantic correction remains underexplored, and LLMs still face over-correction.Source: §2.1 · p. 2

2.2 Text Error Correction Methods Using Reinforcement Learning

  • Correction-related RL work includes GRPO with rule-based rewards, EDGE-GRPO's entropy-driven advantage estimation, reward-confidence stabilization, and CEC-Zero's consensus rewards.Source: §2.2 · pp. 2–3

3 Method

VAA-CSEC frames CSEC as producing natural, semantically correct text while preserving meaning with minimal necessary changes, using a pipeline of CoT distillation, SFT, RL, and self-consistency decoding.

3.1 Overview

  • The pipeline distills CoT rationales, fine-tunes a reasoning model, applies RL, and uses self-consistency decoding at inference.The distilled data comes from a Qwen3.5-27B generator and DeepSeek-V3.2 verifier; SFT initializes the RL stage.Source: §3.1 · p. 3

3.2 Task Definition

  • CSEC transforms potentially erroneous Chinese text into natural, semantically correct text while preserving core meaning and making minimal necessary changes.Source: §3.2 · p. 3

3.3 CoT Distillation

  • CoT distillation pairs each source with a single reference, generates rationales with Qwen3.5-27B, and verifies them with DeepSeek-V3.2, regenerating failures up to three times.Source: §3.3 · p. 3

3.4 Supervised Fine-Tuning

  • SFT on distilled CoT data initializes RL and teaches the structured <think>...</think> <answer>...</answer> format and teacher-derived reasoning trajectories.Source: §3.4 · p. 3

3.5.1 Reward Function

  • The task-specific reward combines format compliance with edit-based correctness, weighting precision more heavily and penalizing excess modification.The format reward requires exactly one <think> block and one <answer> block; correctness uses edit distances to measure useful edits and residual error.Source: §3.5.1 · p. 4, Table 1 · p. 5

3.5.2 The Limitation of GRPO

  • Standard GRPO optimizes expected individual-sample reward, which differs from self-consistency performance determined by which answer dominates the vote.The authors report limited improvement under 32-vote inference and identify this objective mismatch as the reason.Source: §3.5.2 · p. 5

3.5.3 Group-Level Relative Policy Optimization

  • GLPO adds a one-sided vote-margin gain to group-normalized advantages, amplifying samples above the group vote reward while leaving below-vote samples to standard GRPO behavior.The design uses the group vote reward to align optimization more closely with self-consistency.Source: §3.5.3 · p. 5

3.6 Self-Consistency Decoding

  • At inference, the policy generates N stochastic corrections at temperature 1.0 and selects the final correction by majority vote.The experiments vary N across 4, 8, 16, and 32.Source: §3.6 · p. 6

4 Experiments

Experiments evaluate VAA-CSEC on CSED-C and NaSGEC-Exam against Seq2Seq, Seq2Edit, and LLM baselines, using ChER-RANT metrics. VAA-CSEC improves over GRPO and SFT, benefits from self-consistency decoding, and shows cross-domain transfer, though in-domain models achieve the highest F0.5 on each test set.

4.1 Dataset

  • The evaluation uses CSED-C's 10,682 manually annotated sentence pairs and about 7,000 erroneous sentences from NaSGEC-Exam.NaSGEC-Exam includes multiple expert-reviewed reference corrections.Source: §4.1 · p. 6

4.2 Baselines

  • Baselines include four Seq2Seq models, GECToR as a Seq2Edit model, and CSEC-LLM as the LLM-based CSEC method.The Seq2Seq baselines are mT5-small, mT5-base, BART-large-Chinese, and SynGEC.Source: §4.2 · p. 6

4.3 Implementation

  • SFT uses Qwen3.5-4B with LoRA for three epochs, while GLPO training uses LoRA for five epochs on two A30 GPUs.The reported stage runtimes are approximately 4 hours for SFT and 80 hours for RL.Source: §4.3 · p. 6

4.4 Evaluation

  • ChER-RANT reports precision, recall, and F0.5, which weights correction precision more heavily than recall.The paper describes this criterion as consistent with avoiding erroneous modifications and minimizing edits.Source: §4.4 · p. 6

4.5 Main Results

  • 47.72% F0.5 on CSED-C and 41.55% F0.5 on NaSGEC-Exam under 32 Vote make VAA-CSEC stronger than GRPO and SFT on both datasets.On CSED-C, GRPO scores 46.91% and SFT 46.35%; on NaSGEC-Exam, GRPO scores 40.55% and SFT 41.09%.Source: §4.5 · p. 7, Table 3 · p. 7
  • 32 Vote improves VAA-CSEC's F0.5 over greedy decoding by 9.33% on CSED-C and 14.66% on NaSGEC-Exam.The authors report that 32 votes perform best, while inference cost increases exponentially; they recommend 16 votes for practical use.Source: §4.5 · p. 7, Table 4 · p. 7

4.6 Ablation Study

  • Removing CoT distillation causes the largest performance drop, 4.49%; replacing GLPO with GRPO lowers performance by 0.81%.The GLPO replacement decreases recall by 1.27% and precision by 0.63%; removing both components yields 43.64% F0.5 versus 43.23% without CoT distillation alone.Source: §4.6 · pp. 7–8, Table 5 · p. 8

4.7 Cross-Domain Evaluation

  • Training on CSED-C yields higher recall, while training on NaSGEC-Exam yields higher precision; each in-domain model still achieves the highest F0.5 on its test set.The reported in-domain F0.5 scores under 32 Vote are 47.72% on CSED-C and 41.55% on NaSGEC-Exam.Source: §4.7 · p. 8, Table 6 · p. 8

5 Conclusion

VAA-CSEC combines a minimal-editing reward with GLPO, which aligns RL training with self-consistency decoding. It outperforms LLM-based baselines on CSED-C and establishes a state of the art on NaSGEC-Exam; the authors also propose GLPO as a potential approach for other tasks using self-consistency.

5 Conclusion

  • VAA-CSEC addresses over-correction with a task-specific reward and aligns RL training with inference-time self-consistency through GLPO.GLPO reallocates advantages based on the margin between each rollout's reward and the vote-aggregated group reward.Source: §5 · p. 8
  • VAA-CSEC outperforms all LLM-based baselines on CSED-C and establishes a new state of the art on NaSGEC-Exam.The conclusion also reports consistent improvements over SFT and GRPO and additional gains from self-consistency decoding.Source: §5 · p. 8
  • The authors expect GLPO's in-group voting consensus as an optimization anchor to apply to other generation tasks that rely on self-consistency decoding.They frame this as a broader methodological observation about CoT's dependence on sampling strategy.Source: §5 · p. 8

Limitations

The study’s limitations concern both the scale of its rollout-size analysis and the scope of its evaluation. Its experiments cover Chinese semantic error correction on CSED-C and NaSGEC-Exam, leaving larger rollout sizes and other languages unvalidated.

Limitations

  • Experiments cover only CSEC on CSED-C and NaSGEC-Exam; applicability to other languages, especially those with different morphology or error patterns, remains unvalidated.The authors describe GLPO as language-agnostic, but its empirical applicability beyond these Chinese datasets has not been tested.Source: Limitations · p. 9
  • The parameter study tests rollout sizes N ∈ {4, 6, 8}, leaving behavior beyond N = 8 an open question.The authors specifically leave open whether GRPO and VAA-CSEC behave differently at larger N.Source: Limitations · p. 9
  • The interaction between training-time rollout size and test-time self-consistency scale remains to be investigated.Source: Limitations · p. 9

Ethics Statement

The study reports using open-access datasets and models, following research ethics, and using ChatGPT and Claude for textual-error identification and paper polishing.

Ethics Statement

  • The datasets and large language models used in the study come from open-access repositories.Source: Ethics Statement · p. 9
  • The authors state that they comply with relevant ethical standards and authorizations and follow established research ethics.Source: Ethics Statement · p. 9
  • ChatGPT and Claude were used to identify textual errors and polish the paper.Source: Ethics Statement · p. 9

A Detailed Examples of Different Tasks

The section distinguishes spelling, grammatical, and semantic correction by the kinds of errors they target, then gives templates for reasoning generation, correction, and trajectory verification.

A Detailed Examples of Different Tasks

  • Semantic error correction targets logical relations, semantic compatibility, and contextual coherence, including incoherence in sentences without spelling or grammatical errors.Spelling correction addresses surface character errors, while grammatical correction targets syntactic well-formedness.Source: A Detailed Examples of Different Tasks · p. 11

B Template

  • The reasoning-trajectory verifier accepts a trajectory only when its derived correction is completely identical to the reference answer.It rejects missed errors, false error identifications, ambiguity, or inconsistency with the reference.Source: B Template · p. 11
  • The SFT and inference template directs the model to identify semantic issues, explain its judgment, make only necessary edits, and avoid rewriting the entire sentence.It also prioritizes preserving core meaning and deleting redundant or conflicting expressions.Source: B Template · p. 11
  • The CoT-distillation template asks for brief analysis of semantic problems, their type, and the minimal modification while preserving the sentence’s core meaning.It requests a reasoning process that leads naturally to the correction and uses <think> and <answer> output tags.Source: B Template · p. 11

A Detailed Examples of Different Tasks

C Dataset

The study evaluates on CSED-C and NaSGEC-Exam, then compares against sequence-to-sequence, edit-based, and LLM-based correction baselines.

C Dataset

  • CSED-C contains 10,682 manually annotated sentence pairs from Chinese examinations, covering subtle semantic and grammatical errors that demand semantic understanding and reasoning.Source: C Dataset · p. 12
  • NaSGEC-Exam contains approximately 7,000 erroneous examination sentences with corrections and multiple expert-reviewed reference answers per sample.Its examples cover common semantic and grammatical errors and reflect real-world erroneous sentences in native Chinese contexts.Source: C Dataset · p. 12

D Baselines

  • The Seq2Seq baselines are mT5-small, mT5-base, BART-large-Chinese, and SynGEC; SynGEC adds dependency-syntax information to BART.Source: D Baselines · pp. 12–13
  • GECToR represents the Seq2Edit baseline, predicting insertion, deletion, and substitution operations to revise the input rather than generating the target sentence directly.Source: D Baselines · p. 13
  • CSEC-LLM is the LLM baseline, using instruction tuning, example selection, and re-scoring to choose among generated corrections.Its example selection retrieves demonstrations using fine-grained semantic information from multiple dimensions.Source: D Baselines · p. 13

E Training Details in RL

GLPO and GRPO exhibit contrasting training dynamics. The case studies show RL correcting a localized near-homophone error and GLPO making the minimal repair for a sentence-level logical conflict.

E Training Details in RL

  • GLPO reward rises rapidly through the first ~1k steps before oscillating, while entropy and generation length fall and stabilize as the policy concentrates on the consensus trajectory.The voting anchor amplifies gradients on consensus responses while retaining residual stochasticity for inference-time voting.Source: E Training Details in RL · pp. 13–14, Figure 6 · p. 14
  • GRPO reward rises steadily, but policy entropy keeps increasing as the model explores correction strategies and mean generation length decreases.The authors attribute the entropy rise to weak relative-ranking gradients when candidates have similar quality, as multiple valid edits can coexist.Source: E Training Details in RL · pp. 13–14, Figure 7 · p. 14

F Case Study

  • For a near-homophone confusion, CSEC-LLM leaves the sentence unchanged, whereas GRPO and GLPO substitute the correct temporal preposition.The case contrasts the supervised baseline with reinforcement-learning methods on a localized distinction.Source: F Case Study · p. 14
  • For a globally incoherent concessive construction, only GLPO deletes the redundant marker, restoring logical consistency while preserving the causal chain.GRPO edits the wrong connective, while CSEC-LLM makes a surface-level rewrite that leaves the underlying conflict unresolved.Source: F Case Study · p. 14
  • The cases illustrate that GLPO favors the minimum-edit repair when multiple plausible revisions exist.The authors contrast this with RL's ability to recover from simple but easily missed errors.Source: F Case Study · p. 14

G Exploration of Data Ratio

The section tests whether mixing CoT and non-CoT training data can combine their decoding strengths. A 1:1 mixture follows reasoning instructions and performs well under both reasoning and direct-answer settings.

G Exploration of Data Ratio

  • A 1:1 mixture of CoT and non-CoT samples lets the model follow instructions about whether to produce reasoning.The comparison varies training-data ratios and reports performance in Table 7.Source: G Exploration of Data Ratio · p. 15, Table 7 · p. 15
  • When instructed to produce CoT, the mixed model outperforms the CoT-only model under greedy decoding and N-vote inference, especially at N = 32, where it also beats the non-CoT-only baseline.The reported advantage is most pronounced at N = 32.Source: G Exploration of Data Ratio · p. 15
  • When instructed to answer directly, the mixed model retains the strong greedy-decoding performance of non-CoT training.The authors report that the mixed model is stronger along both reasoning and direct-answer dimensions, rather than merely a compromise between training schemes.Source: G Exploration of Data Ratio · p. 15

H Parameter Exploration

The rollout-size study finds non-monotonic performance, with N = 8 strongest among the tested settings, and the section outlines how GLPO uses group voting in advantage computation.

H Parameter Exploration

  • Rollout size N sets the candidate-group size for group-normalized advantages and the vote-based bonus; larger groups increase computational cost roughly linearly.The study tests N ∈ {4, 6, 8}.Source: H Parameter Exploration · p. 15
  • At N = 4, F0.5 is 47.5%; it drops to 46.2% at N = 6, then reaches its peak of 47.72% at N = 8.The authors attribute the dip at N = 6 to insufficient candidate diversity and an unstable group baseline.Source: H Parameter Exploration · pp. 15–16
  • Larger rollout sizes and their interaction with the N-vote inference scale remain open questions because GPU memory constraints limited the study to N ≤ 8.The authors leave exploration of these settings to future work.Source: H Parameter Exploration · p. 16

I GLPO Algorithm

  • For each prompt, GLPO scores K sampled responses, obtains a group response by frequency-based voting, and uses that response's reward as a reference signal in advantage computation.This extends GRPO by incorporating a group-level voting reward.Source: I GLPO Algorithm · p. 16

Related papers

Loading 2609.36804v1…