PaperlayerView exact revision on arXiv

Source-linked AI summary

ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation

Yiling Ma, Yilun Zhao, Sihong Wu, Ziyu Chen, Manasi Patwardhan, Arman Cohan

arXiv:2609.09076v1cs.CL

TL;DR

LLM review systems often identify weaknesses without specifying concrete revision steps, motivating actionable review generation. ActReview separates diagnosis from revision guidance, uses rebuttals as latent supervision, and improves actionability and grounding while retaining a technical-accuracy gap relative to strong proprietary LLMs.

  • Problem

    Existing LLM-based review systems predominantly generate descriptive rather than prescriptive feedback, often identifying issues without specifying how to address them.

  • Method

    ActReview combines a dual-task formulation with rebuttal-guided ActReview-40K training data and multi-task SFT followed by GRPO using candidate-aware, weakness-specific rubric rewards.

  • Results

    ActReview outperforms prior specialized review-generation systems on actionability and grounding while remaining competitive with strong prompt-based LLMs.

  • Takeaways & Limitations

    The framework makes feedback more specific and useful for revision, with gains concentrated on revision-oriented dimensions.

  • Takeaways & Limitations

    ActReview-Bench covers rebuttal-resolvable weaknesses, and the framework does not uniformly outperform strong proprietary LLMs on technical accuracy.

Abstract

from arXiv · show

As LLMs are increasingly used for pre-submission self-review, there is growing demand for feedback that not only identifies weaknesses but also guides authors toward concrete revisions. We study this as Actionable Peer-review Generation and decompose it into two subtasks: diagnostic claim generation and revision suggestion generation. We introduce ActReview, a rebuttal-guided post-training framework that connects paper-specific diagnoses to concrete, grounded revision plans. Our central insight is that author rebuttals reveal plausible actions for addressing reviewer concerns and can therefore provide latent supervision for revision-oriented feedback. From real review-rebuttal threads on OpenReview, we construct ActReview-40K by aligning reviewer weaknesses with author responses and grounding the resulting feedback in localized paper evidence. We post-train Qwen3-8B-Base with multi-task supervised fine-tuning followed by GRPO using candidate-aware, weakness-specific rubric rewards. We also introduce ActReview-Bench, a human-curated benchmark of 1,000 instances for evaluating diagnostic quality and revision usefulness. Experiments show that ActReview outperforms prior specialized review-generation models on actionability and grounding while remaining competitive with strong prompt-based LLMs. Human evaluation confirms improved revision usefulness while revealing a remaining gap in technical accuracy, and additional analyses support generalization to held-out papers and robustness across independent judges.

1 Introduction

ActReview frames actionable peer-review generation as separate diagnosis and revision-guidance tasks, using rebuttals as latent supervision to connect paper-specific weaknesses with concrete revisions. It combines multi-task SFT and rubric-guided GRPO, improving revision-oriented performance while retaining a technical-accuracy gap.

  • Motivation: Existing LLM review systems often identify weaknesses without specifying concrete steps for addressing them.This descriptive feedback limits usefulness when authors need actionable revision guidance.
  • Problem formulation: Actionable Peer-review Generation separates diagnostic claim generation from actionable suggestion generation.The formulation first identifies a concrete paper deficiency, then explains how it should be revised in a grounded and implementable way.
  • Resources: ActReview-Bench evaluates diagnostic quality and practical revision usefulness using rebuttal-derived author actions as grounded reference signals.The benchmark is human-curated and grounded in real review–rebuttal threads.
  • Results: ActReview improves over specialized review-generation models, remains competitive with strong prompt-based LLMs, and shows stronger revision-oriented performance in human evaluation.Human evaluation also identifies remaining challenges in technical accuracy, while further analyses support held-out-paper generalization and judge robustness.
  • Training data: ActReview-40K aligns reviewer weaknesses with rebuttal spans and converts them into structured initial-review-style feedback.Rebuttals infer plausible revision paths but are not presented as post-submission context in the resulting feedback.
  • Training: ActReview combines multi-task SFT with GRPO using candidate-aware, weakness-specific rubric rewards for diagnostic precision, grounding, and revision usefulness.The rubrics encourage connections between paper-specific weaknesses and concrete revision plans.

2 Related Work

Prior work addresses peer-review assistance and open-ended post-training, but ActReview distinguishes diagnosis from revision guidance and constructs structured supervision from review–rebuttal interactions.

  • Peer-review generation and assistance: Peer-review assistance research spans review, rebuttal, interaction, scoring, weakness discovery, and critique-generation tasks.These systems may be prompt-driven, agent-based, trained, or prompted to produce partial or full reviews.
  • ActReview distinction: ActReview separates reviewer feedback into diagnostic claim generation and revision suggestion generation.This explicitly distinguishes detecting a paper-specific weakness from translating it into a concrete revision plan.
  • Post-training: Open-ended post-training methods address tasks where exact-match rewards are unavailable and scalar preferences may be coarse or expensive.Related approaches use LLM evaluators, self-rewarding, and reference-based judges for multidimensional quality assessment.

3 Actionable Peer-Review Generation

The paper formulates actionable review generation as weakness-conditioned diagnosis followed by revision guidance, with abstention when queried weaknesses lack supporting evidence. It builds evaluation around rebuttal-derived actions while acknowledging that multiple revision paths may be valid.

  • Problem formulation: The input combines paper context C with a target weakness label w drawn from a two-level weakness taxonomy.Paper context includes metadata and retrieved paper chunks.
  • Diagnostic claim generation: Diagnostic claims identify concrete paper deficiencies supported by context, while unsupported weaknesses should produce no claim.The model is explicitly permitted to abstain rather than hallucinate a diagnosis.
  • Revision suggestion generation: Actionable suggestions specify what to revise, where it should appear, how to implement it, and the expected outcome.Suggestions are generated for a diagnostic claim under the queried weakness and paper context.
  • Task decomposition: Separating the tasks allows independent training and evaluation while supporting an end-to-end review-to-revision workflow.A controlled comparison reports better suggestion quality and claim–suggestion alignment for the separated formulation than joint end-to-end generation.
  • Inference and evaluation: Weakness-conditioned inference queries one label at a time and can audit a full paper by running independently across taxonomy labels.ActReview-40K and the main benchmark contain only reviewer-raised weakness–paper pairs; a separate held-out set tests zero-shot abstention.
  • Benchmark construction: ActReview-Bench uses rebuttal-derived author actions as grounded reference signals rather than unique gold answers.A rebuttal may represent one plausible revision path, while alternative actionable suggestions can also be valid.

4 ActReview-40K

ActReview-40K converts real review–rebuttal threads into structured, localized training instances through weakness extraction, rebuttal alignment, feedback enhancement, and evidence retrieval. Human validation supports the quality of alignment and filtering, while training uses localized evidence rather than full-paper context.

  • Pipeline: ActReview-40K construction has four stages: atomic weakness extraction, weakness–rebuttal alignment, feedback enhancement, and localized evidence retrieval.Each thread becomes structured initial-review-style feedback for the two subtasks.
  • Data sources: The dataset contains approximately 40K weakness–response instances from 15,819 papers collected across ICLR, NeurIPS, and EMNLP.Weaknesses are assigned to a two-level taxonomy induced from review data and refined by humans.
  • Weakness alignment: Atomic decomposition enables fine-grained alignment between individual reviewer concerns and substantively addressing rebuttal spans.Candidate retrieval uses structural and lexical cues, followed by LLM-based semantic alignment.
  • Validation: Human validation confirms strong alignment quality on a stratified sample.Alignment is evaluated against adjudicated gold spans using token-level IoU ≥0.5.
  • Feedback enhancement: Rebuttal spans provide latent supervision for inferring plausible revision actions, while generated feedback remains in the initial-review perspective.The feedback must not mention rebuttals, author responses, or post-submission changes.
  • Filtering: κ = 0.78 and an 84.8% overall acceptability rate were obtained in a human audit of 500 enhanced instances.Automatic filters remove rebuttal leakage, retrospective rewrites, near-copying, and underspecified suggestions.
  • Evidence retrieval: Training retrieves localized paragraph-level evidence with page and section metadata instead of using the full paper as context.Task 1 supports diagnosis, while Task 2 uses evidence-support and revision-support channels.

5 Post-Training Framework

ActReview trains a unified model in two stages: SFT establishes the dual-task reviewer format, while GRPO uses instance-specific rubrics to improve precision, grounding, and revision usefulness.

  • Training procedure: ActReview-40K is partitioned into 90% SFT data and 10% reinforcement-learning data.Reference claims and suggestions are used offline to construct frozen training rewards rather than as direct RL supervision.
  • Multi-task SFT: SFT jointly trains diagnostic claims and revision suggestions in a unified instruction format.Task 1 maps weakness labels and retrieved paper chunks to diagnoses; Task 2 maps diagnoses and paper context to structured suggestions.
  • Rubric-based GRPO: Candidate-aware weakness-specific rubrics assess instance-level diagnostic precision, grounding, and revision usefulness.The rubrics combine diverse SFT outputs with human-written and GPT-5.4-generated references before being frozen for reinforcement learning.
  • Rubric-based GRPO: GRPO samples eight responses per prompt and scores them with frozen candidate-aware rubrics.Hard-constraint violations receive zero reward, while valid outputs receive weighted soft-requirement scores.

6 Experiments

Experiments evaluate ActReview against prompted and specialized systems using automatic metrics, pairwise judge comparisons, human judgments, and component ablations. Results favor ActReview on actionability, grounding, and trained-model reference metrics, while technical accuracy remains competitive with frontier models and localized context has cross-section limits.

  • Evaluation setup: ActReview is evaluated against prompt-based LLMs and specialized review-generation systems across four experimental questions.The study tests comparative quality, human support for LLM judgments, automatic metrics, and component contributions.
  • Automatic evaluation: 1.24 claims per instance brings ActReview-RL closer to the 1.37-claim reference average than over-generating baselines.For Task 1, ActReview-RL also improves ROUGE-L, BLEU, semantic similarity, specificity, and rubric score.
  • Automatic evaluation: ActReview-RL achieves the strongest trained-model results on Task 2 ROUGE-L, BLEU, suggestion similarity, evidence specificity, and rubric score.These metrics indicate closer alignment with reference revision guidance and paper-specific evidence.
  • Pairwise evaluation: Human and GPT-5.4 pairwise evaluations especially favor ActReview-RL over specialized baselines on Task 2 actionability and overall quality.Against GPT-5.1, Gemini, and schema-controlled variants, preferences are closer to parity, and technical accuracy is the most competitive dimension.
  • Ablations and robustness: Rebuttal-enhanced data yields particularly large Task 2 gains, while localized context remains competitive with full-paper training overall.Full-paper training is preferable for concerns requiring cross-section reasoning, and held-out-paper analyses preserve usefulness gains.
  • Ablations: The two-task formulation improves Task 2 Overall by 0.42 points and Claim–Suggestion Alignment by 0.35 points over end-to-end SFT.Candidate-aware rubrics also outperform direct judge rewards and fixed task-level rubrics.

7 Conclusion

ActReview separates diagnosis from revision guidance and uses rebuttal-derived latent supervision plus candidate-aware rewards to generate actionable, grounded feedback. Its gains are strongest for actionability and grounding, while the results support rebuttals and task separation as useful supervision sources.

  • Contributions: ActReview separates diagnostic claim generation from revision suggestion generation in a Review2Revise framework.The conclusion reports that this structure produces more targeted and executable feedback than treating review generation as one task.
  • Findings: ActReview consistently outperforms specialized review-generation systems on actionability and grounding while remaining competitive with strong prompt-based LLMs.Ablations show that rebuttal-guided enhancement and candidate-aware rubric rewards each contribute independently to output quality.

Limitations

The paper’s scope is bounded by rebuttal-resolvable weaknesses, uneven technical-accuracy gains against proprietary LLMs, and reliance on GPT-5.4 throughout several pipeline stages.

  • Scope: ActReview-Bench excludes novelty disputes, deep conceptual disagreements, and irreparable methodological flaws that lack concrete author-side revision actions.The benchmark is focused on weaknesses that can be linked to rebuttal-resolvable revision paths.
  • Technical accuracy: ActReview improves grounding and actionability but does not uniformly outperform strong proprietary LLMs on technical accuracy.The paper identifies deeper technical judgment as an important direction for future work.
  • Evaluation dependence: GPT-5.4 is used for data enhancement, rubric construction, semantic reward scoring, and scalable pairwise evaluation, creating judge-model coupling risk.The authors mitigate this risk with offline rubric construction, disjoint evaluation instances, matched-context comparisons, and human evaluation.

A Benchmark Annotation Details

ActReview-Bench retains review–rebuttal instances only when author responses directly address concerns, propose concrete revisions, and are supported by discussion when available. Independent annotation and adjudication operationalize these criteria and distinguish reliable revision signals from unresolved or merely fluent responses.

  • Benchmark curation treats rebuttal-derived actions as grounded reference signals, not unique gold answers, because multiple revision paths may address one concern.
  • Each candidate is judged for substantive relevance, concrete actionability, and discussion-level support when follow-up is available.
  • Two annotators independently review the concern, rebuttal, and available follow-up before making criterion-level judgments and a keep/filter decision.
  • Criterion judgments use a 2/1/0 scale for clear yes, borderline, and no, with N/A allowed when discussion-level evidence is unavailable.
  • Retained examples satisfy all three criteria, whereas filtered cases may lack a concrete action or remain unresolved in reviewer follow-up.
  • The benchmark focuses on rebuttal-resolvable weaknesses and excludes unresolved novelty disputes, fundamental disagreements, and concerns without a convergent revision path.

B.2 Data Licensing and Attribution

ActReview-40K uses publicly available review–rebuttal and paper sources while preserving attribution and respecting licensing constraints. The data supports structured extraction, localized retrieval, and review-feedback construction rather than ownership of original content.

  • ACTREVIEW-40K draws review comments, rebuttals, discussion threads, and metadata from OpenReview under CC BY 4.0, while paper licenses vary across repositories.
  • The authors do not claim ownership of original papers or review content and use it for structured extraction, localized retrieval, and context construction.
  • The source material is publicly available, but the release avoids adding non-public identities or private metadata and is intended for research use.
  • The dataset construction transforms review–rebuttal examples into diagnostic claims, evidence, and actionable revision guidance without directly imitating rebuttal text.

C.1 Data-Driven Weakness Taxonomy

The paper builds a human-refined weakness taxonomy and a multi-stage pipeline that aligns atomic review concerns with rebuttal spans, enhances them into structured feedback, and retrieves localized evidence. Audits show strong taxonomy agreement and generally high enhancement quality, while revision grounding and localization remain harder.

  • C.1 Data-Driven Weakness Taxonomy: The taxonomy is induced from roughly 500 sampled weaknesses through LLM-assisted discovery, semantic merging, and human review.
  • C.1 Data-Driven Weakness Taxonomy: Fleiss’ κ reaches 0.79 at Level 2 and 0.88 at Level 1 on 100 independently labeled weaknesses, indicating substantial annotation agreement.
  • C.1 Data-Driven Weakness Taxonomy: The final taxonomy contains 7 Level-1 and 17 Level-2 categories, while reviews are decomposed into atomic weaknesses before rebuttal alignment.
  • C.3 Alignment Quality Audit: ActReview-40K maps each atomic weakness to a rebuttal span, audits 794 mapped segments from 50 papers, and counts span matches using token-level IoU ≥ 0.5.
  • C.4 Constructing Enhanced Instances: Enhanced outputs contain diagnostic claims plus revision suggestions specifying what, where, how, and expected outcomes, with 84.8% audited acceptability and 93.2% concern preservation.
  • C.5 Enhancement Quality Audit: Rebuttal-supported revision guidance has an 86.4% pass rate, with errors arising when indirect responses require unsupported inference or broad locations replace concrete revision sites.
  • C.6 Localized Evidence Retrieval: Localized retrieval remains competitive with full-paper conditioning at substantially lower input cost, although Task 2 localization remains more challenging.
  • C.6 Localized Evidence Retrieval: Task 1 is relatively insensitive to chunk size, whereas Task 2 benefits from additional context and generally plateaus around k=5.

D Rubric Construction and Reward Design

The RL stage uses frozen, candidate-aware rubrics to reward instance-specific diagnostic and revision quality rather than generic advice. Fixed-rubric comparisons and human evaluation frame the design around paper grounding, actionability, and reliable preference measurement.

  • D.1 Candidate-Aware Rubric Construction: Candidate-aware rubrics distinguish the specific missing evidence, paper setting, revision location, and failure modes relevant to each training instance.
  • D.1 Candidate-Aware Rubric Construction: Rubric construction compares diverse SFT outputs with human-style and strong-LLM references, then combines hard constraints with weighted soft requirements.
  • D.1 Candidate-Aware Rubric Construction: Rubrics are built offline and frozen during GRPO; reference outputs help define reward criteria but are not used as direct supervision.
  • D.2 Rubric Scoring Prompt: An LLM judge scores each weighted soft requirement from 1 to 5, normalizes scores to [0, 1], and combines them using rubric weights.
  • D.3 Direct Judge Reward Prompt: Ablations compare candidate-aware rewards with direct-judge scoring and fixed task-level rubrics shared across instances.
  • D.4 Fixed Task-Level Rubric Prompt: Task 1 rubrics assess diagnostic accuracy, specificity and grounding, depth, granularity, format, and overall usefulness.
  • D.4 Fixed Task-Level Rubric Prompt: Task 2 rubrics assess technical accuracy, actionability, grounding and localization, depth, and practical usefulness, while penalizing rebuttal references, unsupported content, wrong labels, generic advice, and format failures.
  • E.2 Human Evaluation Instructions: Human pairwise judgments show 93% identical categorical choices, while disagreements are retained by averaging scores rather than forcing a single winner.

E.3 Automatic Metrics.

The paper combines lightweight automatic metrics, example-specific rubric scoring, claim alignment, and bootstrap intervals to evaluate diagnostic and revision outputs. It also reports a task-decomposition experiment showing stronger overall quality and claim–suggestion alignment.

  • Automatic metrics: Specificity measures localized paper references, technical terminology, and quantitative details in generated text.For Task 1 it is averaged across claims and evaluation instances; for Task 2 it is applied to the extracted evidence section.
  • Automatic metrics: Suggestion similarity measures whether each generated Task 2 suggestion is semantically aligned with at least one reference action using sentence embeddings and cosine similarity.The metric compares generated suggestion blocks with reference suggestion blocks.
  • Automatic metrics: Example-specific Rubric scores outputs against offline, instance-specific rubrics containing hard constraints and weighted soft requirements.Hard-constraint violations receive zero, while otherwise soft requirements are scored from 1–5 and mapped to [0, 1].
  • Claim alignment: Task 1 claims are aligned one-to-one with reference claims by maximizing total semantic similarity before ROUGE-L, BLEU, and semantic similarity are averaged.Unmatched generated and reference claims represent over-generation and under-generation, respectively, with average generated claim count reported separately.
  • Uncertainty and interpretation: 95% bootstrap confidence intervals are computed by resampling evaluation instances and taking empirical 2.5th and 97.5th percentiles.The same instance-level resampling principle is used for automatic metrics and pairwise adjusted win rates.
  • Task-decomposition analysis: The two-task formulation improves all five evaluation dimensions, including +0.42 points in Task 2 Overall quality and +0.35 points in Claim–Suggestion Alignment.The comparison isolates task decomposition before rubric-based reinforcement learning and uses the same backbone, data, optimization settings, and evaluation protocol.

F.2 Independent Evaluation on 2025–2026 Papers

An independent evaluation on 2025–2026 OpenReview papers tests whether ActReview generalizes beyond rebuttal-derived references. ActReview remains competitive on diagnostic quality while providing stronger revision guidance, though broader cross-domain generalization is not established.

  • Evaluation setup: 175 instances from 50 unseen papers were evaluated by independent reviewers without access to the original review–rebuttal discussions.All systems received identical paper context, metadata, and weakness labels, and paired diagnostic and suggestion outputs were evaluated jointly.
  • Results: ACTREVIEW-RL remained comparable to GPT-5.1schema in Weakness Validity (1.68 versus 1.67) and Technical Correctness (1.66 versus 1.64).Its larger gains were in revision-oriented dimensions.
  • Results: Resolution Sufficiency increased from 1.23 to 1.70, and Reviewer Usefulness increased from 1.34 to 1.68, for ACTREVIEW-RL versus GPT-5.1schema.ACTREVIEW-SFT also outperformed proprietary baselines on these two dimensions.
  • Scope: The evaluation does not establish generalization across venues, disciplines, substantially different review norms, or fundamentally non-resolvable concerns.These boundaries include irreparable methodological flaws and purely subjective novelty disputes.
  • Technical quality: ACTREVIEW-RL achieved the highest Valid + Actionable rate at 74%, compared with 55% for GPT-5.1 and 61% for ACTREVIEW-SFT.It also had the lowest Actionable but Invalid rate at 10%, while Scientific Validity was similar to GPT-5.1 (1.71 versus 1.70).
  • Technical quality: ACTREVIEW-RL matched Geminischema at a 28.00% overall technical-error rate and achieved the lowest severe-error rate at 6.29%.Relative to ACTREVIEW-SFT, rubric-based reinforcement learning reduced several unsupported, misunderstanding, recommendation, omission, and severe-error categories.
  • Context analysis: Full-paper training improved Cross-Section Reasoning from 1.17 to 1.49 and reduced Unsupported Claim Rate from 22% to 14%, while localized retrieval retained an advantage in Resolution Sufficiency.The findings indicate complementary benefits rather than universal superiority of either context.
  • Hard cases: In hard cases, ACTREVIEW-RL extended incomplete rebuttals in 46% of cases and rejected inadequate resolution paths in another 20%.It introduced unsupported resolutions in 6% of cases, indicating that the model can depart from author responses while usually remaining grounded.

G.4 Results

The screened non-supported evaluation tests whether models abstain when a queried weakness lacks evidence. ActReview abstains substantially more often than baselines while preserving reliable answers on supported instances, but residual over-generation remains.

  • Supported instances: All models answered supported instances reliably, with Answer Rate at least 95% and False-Abstention Rate at most 5%.This indicates that stronger abstention was not caused by indiscriminate refusal.
  • Non-supported instances: ACTREVIEW-RL abstained on 78.3% of non-supported instances while maintaining a 4.6% False-Abstention Rate on supported instances.ACTREVIEW-SFT abstained on 71.4% of non-supported instances.
  • Reward effects: Reinforcement learning increased abstention over SFT by 6.9 percentage points, consistent with rubric penalties for wrong-label and hallucinated claims.The cited analysis characterizes this as consistent with the grounding-oriented reward design.
  • Limitations: ACTREVIEW-RL still over-generated on 21.7% of screened non-supported pairs.The remaining errors motivate explicit negative training and harder within-category distractor evaluation.
  • Case study: Figure 20 contrasts two correct None outputs with one formatting-related over-generation error inferred from discussion of Table 7.Correct abstentions are shown in green, while distractor labels and the wrong inference are shown in red.
Loading 2609.09076v1…