PaperlayerView exact revision on arXiv

Source-linked AI summary

CriticGen: Generation-Aware Evaluation as Actionable Feedback

Huifang Du, Zecheng Zuo, Sen Wang, Chenghao Fan, Haofen Wang, Yehui Yang

arXiv:2609.05439v1cs.AI

TL;DR

Existing evaluation methods are often coarse-grained and decoupled from generation, leaving a gap between judging answers and improving them. CriticGen induces sample-specific rubrics to connect scoring, explanation, executable revision, and answer rewriting, improving rubric quality, evaluation reliability, and answer refinement.

  • Problem

    Current evaluation methods often use fixed or coarse criteria and remain disconnected from answer improvement, limiting feedback adaptiveness and actionability.

  • Method

    CriticGen induces a sample-specific rubric for each question–answer pair and uses it to jointly produce a score, criterion-grounded reason, executable refinement suggestion, and refined answer.

  • Results

    73.17% of answers improved with a 93.28% non-degradation rate, while rubric relevance/coverage rose from 3.33/4.03 to 3.97/4.24 and score correlations reached 0.9556 Pearson and 0.9560 Spearman.

  • Takeaways & Limitations

    CriticGen shows that evaluation can function as a direct mechanism for controllable generation improvement rather than only as a diagnostic signal.

  • Takeaways & Limitations

    CriticGen depends on supervised fine-tuning data and has so far been evaluated mainly on text reasoning tasks without reinforcement-learning optimization.

Abstract

from arXiv · show

Current evaluation methods for large language models are coarse-grained and decoupled from generation, producing generic explanations that fail to provide actionable feedback for model improvement. We propose CriticGen, a fine-grained, generation-aware evaluation framework that turns evaluation into actionable control for answer improvement. CriticGen first generates sample-specific evaluation dimensions and scoring criteria under high-level categories such as subjective, objective, and self-derived constraints. These criteria then serve as a dynamic rubric for jointly producing a score, a reason, an executable refinement suggestion, and a refined answer. This rubric-conditioned refinement process enables models to diagnose flaws and perform targeted answer improvement. Experimental results show that fine-grained evaluation should be both instance-specific and actionable. CriticGen induces higher-quality rubrics, improving relevance/coverage from 3.33/4.03 to 3.97/4.24. CriticGen also achieves the best score correlations, with 0.9556 Pearson and 0.9560 Spearman, and raises the F1 of criterion-grounded reasons and executable suggestions from 0.6369/0.5994 to 0.7554/0.7900. Crucially, its feedback translates into reliable answer improvement, improving 73.17% of answers with a 93.28% non-degradation rate.

1 Introduction

CriticGen addresses coarse, fixed, and generation-decoupled evaluation by inducing sample-specific rubrics and using them to connect diagnosis with executable answer refinement. Its results support fine-grained evaluation that is both instance-specific and actionable.

  • Motivation: Current methods often use coarse or fixed criteria and separate evaluation from generation, leaving feedback insufficiently actionable for improving the same answer.Existing evaluation may indicate quality or flaws without specifying where, why, or how to revise the answer.
  • Method: CriticGen induces a sample-specific rubric with dimensions and graded criteria organized under subjective quality, objective correctness, and self-derived reasoning constraints.The rubric is specific to the current question–answer pair rather than fixed at the task or scenario level.
  • Method: CriticGen uses the rubric to jointly produce a score, criterion-grounded reason, executable refinement suggestion, and refined answer.This connects what to evaluate, why the answer is flawed, how to revise it, and how to generate the revision.
  • Method: CriticGen trains separate rubric-generation and evaluation-refinement components with RUBRICDATA and REFINEDATA, without a separate reinforcement-learning stage.The framework studies whether structured supervised trajectories can directly provide an executable refinement signal.
  • Results: 3.33/4.03 to 3.97/4.24: human-rated rubric relevance/coverage improves over a static checklist.The reported relevance/coverage values are the before-and-after scores for CriticGen’s induced rubrics.
  • Results: 0.9556 Pearson and 0.9560 Spearman: CriticGen-Qwen3.5-9B achieves the best score correlations, while reason and suggestion F1 rise to 0.7554/0.7900.The reason and suggestion F1 values improve from the best baseline’s 0.6369/0.5994.
  • Results: 73.17% of answers improve with a 93.28% non-degradation rate, and out-of-domain results show consistent transfer of rubric-conditioned evaluation ability.Non-degradation counts improved and unchanged cases, while the transfer result is reported on Feedback Bench.

2 Data Construction

CriticGen’s data construction supports two related capabilities: inducing instance-specific rubrics and refining answers under each rubric item. The pipeline combines diverse reasoning answers, generated supervision, score coverage completion, consensus filtering, and deduplication.

  • Data requirements: CriticGen requires separate supervision for generating instance-specific rubrics and refining answers under a given rubric.The two capabilities are supported through distinct but related training-data construction processes.
  • Answer pool: The diverse answer pool covers arithmetic, symbolic/logical, open-domain natural-language, competition-level math, and long-form chain-of-thought reasoning.Questions come from GSM8K, LIMO, NaturalReasoning, NuminaMath-CoT, and QwQ-LongCoT-130K.
  • Rubric supervision: For each question–answer pair, rubric supervision stores sample-specific dimensions and 0–5 scoring criteria under subjective quality, objective correctness, and self-derived constraints.These annotations form supervision for rubric learning.
  • Refinement supervision: For each rubric item, evaluation-refinement supervision includes a score, criterion-grounded reason, executable refinement suggestion, and refined answer.The score and reason capture criterion satisfaction, while the suggestion and refined answer instantiate concrete improvement.
  • Quality coverage: Missing regions of the 0–5 quality range are filled with synthesized answers, followed by repeated rubric and evaluation-refinement supervision generation.This score-coverage step also increases answer diversity beyond the initial answer pool.
  • Filtering and splits: Samples are retained when independent model scores agree with the GPT-5 score, then filtered for lexical diversity and near-duplicate supervision.The final RUBRICDATA and REFINEDATA splits use 5:1 and 50:1 training-to-test ratios, respectively.

3 Method

CriticGen separates sample-specific rubric induction from rubric-conditioned evaluation and refinement, using structured criteria to connect diagnosis with executable answer improvement.

  • Rubric-Conditioned Evaluation and Refinement: The structured formulation makes evaluation descriptive through scores and reasons, but operational through refinement suggestions and revised answers.This connects what to evaluate, why an answer is flawed, how it should change, and how to generate the revision.
  • Sample-Specific Rubric Induction: CriticGen induces a dynamic rubric for each question–answer pair, specifying evaluation dimensions and graded criteria under broad constraint families.The families cover subjective quality, objective correctness, and self-derived reasoning constraints.
  • Rubric-Conditioned Evaluation and Refinement: For each rubric item, the model produces a score, criterion-grounded reason, executable refinement suggestion, and refined answer.The suggestion converts the diagnosed weakness into a concrete edit operation and target content.
  • Training: CriticGen trains its two components with supervised fine-tuning rather than a separate reward or preference-optimization stage.The supervision consists of instance-specific rubrics and rubric-conditioned evaluation-to-refinement trajectories.
  • Training: The rubric generator is trained on constructed question–answer rubrics, while the refinement model learns serialized score–reason–suggestion–rewrite outputs.For refinement training, retained scores are consensus-filtered when multiple teacher models agree.
  • Framework Integration: Although trained independently, the modules are coupled at inference through the induced rubric, which interfaces evaluation-target generation with actionable refinement.Figure 2 summarizes this flow from sample-specific rubric induction to unified structured outputs.

4 Experiments

Experiments test sample-specific rubric induction and rubric-conditioned evaluation, generation, and refinement. CriticGen improves rubric quality, score agreement, criterion-grounded feedback, and answer-refinement reliability.

  • Experimental setup: Experiments evaluate sample-specific rubric induction and rubric-conditioned scoring, reasons, executable suggestions, and answer refinement.The evaluation uses held-out data, human rubric judgments, reference-score correlations, semantic-unit alignment, and verifier-based refinement outcomes.
  • Rubric quality: 3.97/4.24 versus 3.33/4.03: CriticGen improves human-rated rubric relevance/coverage over Static.The evaluation covers 240 held-out question–answer pairs, with higher gains on relevance than coverage.
  • Evaluation capability: 0.9556/0.9560 Pearson/Spearman: CriticGen-Qwen3.5-9B achieves the strongest score correlations.It surpasses Qwen3.5-9B and GPT-5, while smaller CriticGen variants also show large gains over their backbones.
  • Evaluation capability: 0.7554 reason F1 and 0.7900 executable-suggestion F1 exceed the best baseline values of 0.6369 and 0.5994.Executable suggestions show the larger gain, with recall increasing to 0.7921.
  • Refinement capability: 73.17% of cases improve and 93.28% avoid degradation under CriticGen refinement.The outcome distribution is 73.17% improved, 20.11% unchanged, and 6.72% degraded; the non-degradation rate sums improved and unchanged cases.
  • Ablation study: 73.54 improved rate: Full CriticGen outperforms variants removing reason or suggestion supervision and the rewrite-only variant.Removing reason or suggestion supervision lowers refinement success to 60.48 and 61.57, while rewrite-only reaches 62.35.

5 Conclusion

CriticGen turns fine-grained evaluation into actionable control by inducing sample-specific rubrics and jointly generating evaluation and refinement outputs. Experiments report improvements in rubric quality, evaluation quality, and refinement reliability.

  • Conclusion: CriticGen induces sample-specific rubrics and uses them to jointly generate a score, reason, executable suggestion, and refined answer.The framework connects diagnosis and revision through rubric-conditioned generation.
  • Conclusion: CriticGen improves rubric quality, score correlation, reason and suggestion generation, and answer-refinement reliability.The reported results support evaluation as both a diagnostic signal and a mechanism for controllable generation improvement.

Limitations

The paper identifies supervision quality, domain coverage, and the absence of reinforcement-learning optimization as limitations. Current validation focuses on text reasoning tasks and structured supervised fine-tuning.

  • Scope and limitations: Supervised fine-tuning makes CriticGen dependent on the quality and coverage of constructed supervision data.Generated rubrics and refinement trajectories may introduce inherited biases or blind spots.
  • Scope and limitations: Experiments focus on text reasoning tasks, leaving multimodal reasoning, long-form generation, and interactive decisionmaking for further validation.The paper explicitly identifies these domains as broader settings requiring additional evaluation.
  • Scope and limitations: CriticGen does not yet optimize its model with reinforcement learning or preference optimization.The current framework directly converts evaluation into refinement through a structured supervised-fine-tuning trajectory.

B Data Filtering

The constructed data are filtered for lexical diversity and near-duplicate removal across input-side and generated fields. These steps preserve meaningful variation, especially in reasons, executable suggestions, and refined answers.

  • Field coverage: Lexical statistics cover questions, original answers, rubrics, reasons, executable suggestions, and refined answers after normalization into word-level tokens.Input-side and generated fields are analyzed separately because they serve different roles in the evaluation-to-refinement trajectory.
  • Lexical diversity: Distinct-n measures the ratio of unique n-grams to total n-grams, with higher values indicating greater diversity and lower template reuse.Distinct-2, Distinct-3, and Distinct-4 are computed for each field collection.
  • Near-duplicate filtering: Candidate instances are filtered by concatenating generated fields and removing those whose maximum 4-gram Jaccard overlap exceeds τ = 0.75.MinHash-based locality-sensitive hashing retrieves likely overlaps before exact Jaccard computation.
  • Outcome: The retained data show meaningful lexical variation across input-side and generated fields rather than repeated templates.Questions are more standardized, whereas reasons, executable suggestions, and refined answers exhibit high-order diversity.

C Answer Sensitivity of Induced Rubrics

Answer-sensitivity is measured by comparing induced rubrics for different answers to the same held-out question. CriticGen produces substantially more rubric change than question-only controls.

  • Data and protocol: The analysis uses 180 held-out questions with 3–5 candidate answers each, reserved exclusively for answer-sensitivity evaluation.The questions are held out from training.
  • Data and protocol: CriticGen compares answer-conditioned rubrics R(q, a) and R(q, a′), while Q-only uses one shared question-only rubric and Q-only×2 controls repeated-generation variation.The protocol tests whether induced rubrics change when the answer changes.
  • Measurement: Rubric turnover is computed from answer-conditioned changes in the set of lowercased alphanumeric word types appearing in dimension_name strings.The analysis excludes 0–5 criterion text to reduce sensitivity to long paraphrases in scoring anchors.
  • Results: 0.37 is CriticGen’s mean rubric shift, versus 0 for Q-only and 0.09 for Q-only×2.The larger shift supports answer sensitivity beyond lexical variation from repeated question-only generation.

D Baseline Models and CriticGen Variants

The experiments compare CriticGen variants with open-weight and proprietary instruction-following baselines across model families and scales. The variants use corresponding Qwen backbones and the constructed training data.

  • Baseline models: The baseline set includes Qwen, Mistral, and GPT-5 models to compare CriticGen with contemporary open-weight and proprietary instruction-following systems.GPT-5 also serves as a reference judge in parts of data construction and verification.
  • Baseline models: Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, and Qwen2.5-32B-Instruct provide scale-controlled Qwen2.5 baselines.These comparisons examine whether capacity alone improves rubric-conditioned evaluation and refinement.
  • Baseline models: Qwen3.5-4B and Qwen3.5-9B test rubric-conditioned evaluation with newer base models across parameter scales.They assess whether recent base-model improvements support the task without task-specific fine-tuning.
  • CriticGen variants: CriticGen-Qwen2.5-7B-SFT, CriticGen-Qwen3.5-4B-SFT, and CriticGen-Qwen3.5-9B-SFT apply the same supervised fine-tuning procedure to corresponding Qwen backbones.They are trained on RUBRICDATA and REFINEDATA to isolate the effects of CriticGen’s data and learning objective from backbone choice.

E Training Details

CriticGen is trained with supervised fine-tuning on serialized rubric-induction and rubric-conditioned evaluation-to-refinement targets. The main experiments use SFT checkpoints without an additional preference-optimization stage.

  • Supervised fine-tuning: All CriticGen training runs use supervised fine-tuning from instruction-tuned backbones with a shared serialized output format.Rubric induction targets include dimension names and 0–5 scoring criteria.
  • Training targets: Rubric-conditioned training targets contain the score, reason, executable revision suggestion, and refined answer.This trains the model to produce the complete evaluation-to-refinement sequence.
  • Checkpoint selection: The selected SFT checkpoint is used as the main CriticGen model without preference optimization.This keeps comparisons focused on the proposed data construction and evaluation-to-refinement formulation.

F Out-of-Domain Evaluation on Feedback Bench

CriticGen is evaluated on Feedback Bench, an independently constructed out-of-domain benchmark requiring rubric-conditioned scoring and feedback. It achieves the best Pearson, Spearman, and Kendall’s τ correlations, with consistent gains over its base model.

  • Benchmark and setup: Feedback Bench evaluates scalar scores and feedback against customized, fine-grained rubrics for each instance.Each instance includes an instruction, response, reference answer, and scoring rubric.
  • Benchmark and setup: CriticGen is tested out of domain because Feedback Bench was not used during training and follows an independently constructed evaluation format.The comparison uses similarly scaled evaluator models, including Qwen2.5-7B-Instruct, Qwen3.5-4B, and Mistral-7B-Instruct-v0.3.
  • Evaluation protocol: Evaluation measures score agreement with reference scores using Pearson correlation, Spearman correlation, and Kendall’s τ.The protocol follows Prometheus’s evaluation procedure.
  • Results: CriticGen-Qwen2.5-7B-SFT achieves the best performance across all three correlation metrics.Compared with Qwen2.5-7B-Instruct, Pearson rises from 0.7520 to 0.7556, Spearman from 0.7522 to 0.7612, and Kendall’s τ from 0.6606 to 0.6755.
  • Results: The consistent gains suggest that CriticGen’s rubric-conditioned evaluation capability transfers to out-of-domain customized scoring settings.The reported improvements are modest but consistent across Pearson, Spearman, and Kendall’s τ.
Loading 2609.05439v1…