PaperlayerView exact revision on arXiv

Source-linked AI summary

PeerCheck: Enhancing LLM-Generated Academic Reviews Towards Human-Level Quality

Zeyuan Chen, Ziqing Yang, Yihan Ma, Michael Backes, Yang Zhang

arXiv:2606.20897v1cs.CLcs.AI

TL;DR

Peer review faces growing submission pressure, and LLM assistance raises questions about review quality and alignment. PeerCheck compares human and LLM reviews and tests prompting and retrieval approaches; it reports improved detection and similarity outcomes, alongside model-dependent RAG effects.

  • Problem

    Growing submissions and limited reviewer availability challenge peer review, while LLM assistance raises questions about review quality and alignment with scholarly standards.

    Source: §1 · p. 1

  • Method

    PeerCheck compares human-written and LLM-generated reviews, then uses prompt engineering, Chain-of-Thought prompting, and RAG to improve generated reviews.

    Source: §1 · p. 1, §3 · p. 2

  • Results

    40.3% reduction in LLM-generated review detection rates and 26.6% increase in similarity to human-written reviews; RAG effects vary by model.

    Source: §1 · p. 2, §7 · p. 7

  • Takeaways & Limitations

    PeerCheck is intended as an assistive analytical tool, not an automated reviewer or decision-making system.

    Source: Ethical Consideration · p. 8

    “Human-level quality” is operationalized as lexical and semantic similarity to historical human reviews, which does not capture novel insight or constructive mentorship.

    Source: Ethical Consideration · p. 8

Abstract

from arXiv · show

As academic submissions grow, the traditional peer review process struggles to keep up, raising concerns about quality and fairness. A trend of using large language models (LLMs) for assistance has emerged. In this work, we take a critical step toward improving the quality of LLM-generated reviews. We propose the PeerCheck framework, which investigates LLM-human review differences (RQ1) and explores methods to improve LLM-generated review quality (RQ2). We first analyzed the human-written reviews with reviews generated by various LLMs and found that LLMs and humans focus on different terms, e.g., LLMs prioritize theory while humans emphasize methodology and experiments. We further adopt prompt engineering, such as Chain-of-Thought (CoT), and utilize retrieval-augmented generation (RAG) to enhance the LLM-generated reviews towards human-level quality. We find CoT significantly improves the quality of LLM reviews, while we discover an unexpected "RAG paradox," i.e., experiments with RAG produce different results for various LLMs and, in some cases, even reduce review quality. Our comprehensive analysis of LLM-generated academic reviews illustrates both possibilities and limitations, contributing to a more effective, human-aligned review system. Our dataset is available on https://github.com/TrustAIRLab/PeerCheck.

1 Introduction

Peer review faces growing submission volumes and limited reviewer availability, while LLM assistance raises questions about review quality and alignment. PeerCheck compares human and LLM reviews and tests ways to improve LLM-generated reviews.

Our Contribution

  • The paper asks how LLM-generated reviews differ from human reviews and how their quality can be improved toward human-level standards.These are its two research questions.Source: §1 · p. 1
  • PeerCheck reports GPT-4o’s human-like score rising from 0.379 to 0.645.The framework compares human and LLM reviews, then applies prompt engineering and RAG to improve generated reviews.Source: §1 · pp. 1–2
  • The RAG paradox is model-dependent: retrieval improves GPT-4o but degrades Claude’s performance.This challenges the assumption that more information always improves reviews.Source: §1 · p. 2
  • Claude-3.7-Sonnet assigns scores of 8 at a 57.7% higher rate in the “PhD student” role than with no specified role.PeerCheck identifies this as role sensitivity in LLM-generated reviews.Source: §1 · p. 2
  • Revised Chain-of-Thought prompts reduce the RoBERTa-based LLM-review detection rate by 40.3% from 0.878.The paper presents this reduction as enhancing review authenticity.Source: §1 · p. 2

2 Background and Related Works

Prior work explores LLMs for academic review and methods for evaluating or detecting generated text, but standardized comparisons across advanced models and links between detection and improvement remain limited.

LLMs in Academic Review

  • Prior studies report overlap between human and LLM review points, while LLMs miss subtle flaws requiring expertise.One cited study found LLMs identify 68% of methodological issues caught by humans.Source: §2 · p. 2
  • Comparisons across advanced models using standardized review criteria remain limited, and many approaches do not target academic-review weaknesses.PeerCheck positions its quantitative comparisons and targeted enhancement strategies as addressing these gaps.Source: §2 · p. 2

LLM-Generated Content Evaluation and Detection

  • Detection research includes multi-feature and watermarking methods, while current tools are described as neither accurate nor reliable.Integrating detection insights with improvement strategies remains underexplored for academic review.Source: §2 · p. 2
  • LLM reviews can show generic criticism and limited expertise signaling, characteristics identified as making them detectable.PeerCheck investigates human–LLM differences and revised prompts to address them.Source: §2 · p. 2

3 PeerCheck

PeerCheck collects academic papers and reviews, compares human and LLM review patterns, then develops and validates revised prompts, including Chain-of-Thought and retrieval-augmented approaches.

Review Collection

  • PeerCheck processes 1,920 papers with GPT-4o, Claude-3.7-Sonnet, and DeepSeek-V3 to generate benchmark reviews and compare them with official reviews.The generated reviews include strengths, weaknesses, comments or questions, and a final rating.Source: §3 · p. 2

Analysis & Prompt Engineering

  • The Chain-of-Thought process uses three reflective phases to help understand and review manuscripts.PeerCheck uses iterative CoT refinement to guide LLMs through structured evaluation.Source: Figure 2 · p. 3, §3 · p. 3
  • Keyword, rating-score, and topic comparisons identify differences between human and LLM reviews that inform revised prompts.The analysis compares distinctive keywords, highly rated papers, and scoring patterns; prompts incorporate human-review language, reasoning styles, reviewer personas, scoring justifications, and disciplinary context.Source: §3 · p. 3

Evaluation and Validation

  • RAG retrieves external knowledge from human reviews to address identified gaps between LLM-generated and human-written evaluations.The validation set contains 480 papers distinct from those used in earlier phases.Source: §3 · p. 3
  • Validation measures LLM-review detectability and lexical and semantic similarity to human-written reviews using metric- and model-based detection methods.The comparison uses a human-review corpus of the same size as the generated-review corpus.Source: §3 · p. 3

4 Experimental Setup

The experiments use conference papers and human reviews to compare three LLMs, with separate analysis and validation sets and multiple detection and similarity measures.

Datasets

  • The study covers ICLR 2024/2025 and NeurIPS 2024 papers, retaining reviews with more than 80% RoBERTa probability of human authorship.An 8:2 split yields 1,920 analysis papers and 480 validation papers.Source: §4 · p. 3

Evaluated LLMs

  • The three evaluated models are GPT-4o, Claude 3.7 Sonnet, and DeepSeek-V3.The paper describes them as representing different LLM designs and training methods.Source: §4 · p. 3

LLM Detection and Similarity Metrics

  • Figure 4 compares LLM and human rating scores in NeurIPS, while the study measures detection and similarity with multiple metrics.Detection metrics include log-likelihood, log-rank, GLTR, LRR, and RoBERTa; similarity metrics include BLEU, ROUGE, token-level F1, and cosine scores.Source: Figure 4 · p. 4, §4 · p. 4

5 Human vs. LLM Reviews

Human and LLM reviews differ in scoring patterns and evaluation priorities: LLMs emphasize theory, while humans stress methods and experiments. Persona-based scoring also varies by model.

Overall Performance Comparison

  • LLMs show broader score distributions than humans and consistently assign higher scores to rejected papers; DeepSeek-V3 rates both accepted and rejected papers higher.Scoring patterns also vary among the LLMs when evaluating rejected manuscripts.Source: §5 · p. 4

Feature Space Distribution

  • Figure 5 shows substantial overlap between human and LLM reviews, alongside isolated Claude-3.7-Sonnet and GPT-4o clusters with distinctive signatures.The visualization uses 400 comments from each source.Source: §5 · p. 4, Figure 5 · p. 4

Topical Analysis

  • Reviewers share attention to large language models and reinforcement learning, but humans emphasize NLP while LLMs favor representation learning and in-context learning.Humans’ modest preference for papers with larger author teams has attenuated across recent conferences.Source: §5 · p. 4

Keywords Analysis

  • Among the top 15 common terms in reviews, all LLM reviews prioritize “theoretical” as their primary criterion, while human reviewers predominantly emphasize “methods” considerations.Human reviewers also emphasize “performance” metrics and “experiments” validation; only humans include “algorithm” and “baseline” among the top 15 keywords.Source: §5 · pp. 4–5

Influence of Role-Play

  • Claude-3.7-Sonnet assigns 8-point ratings 57.7% of the time as a PhD student and 53.5% as a professor; GPT-4o reaches 51.3% as an industry expert, while DeepSeek-V3 peaks at 20.5%.Figure 6 compares persona-based increases in 8-score ratings against the Normal role.Source: §5 · p. 5, Figure 6 · p. 5

Takeaways for RQ1

  • Despite topical overlap, LLMs overemphasize theory and undervalue methods and experiments, identifying recalibration toward human review criteria as an improvement pathway.Source: §5 · p. 5

6 Enhancing LLM-Generated Reviews

Revised prompting improves review similarity and quality, while retrieval-augmented generation (RAG) has model-dependent effects. The results also favor structured evaluation guidance over stylistic imitation.

LLM-Generated Detection

  • Revised Chain-of-Thought prompts reduce LLM-review detection rates by 40.3%.Detection rates vary across models and detection methods; the reported comparison includes five detectors.Source: §6 · p. 5

Similarity With Human Reviews

  • GPT-4o reaches a 0.645 cosine similarity with human reviews using revised prompts with keywords and sample sentences.Claude-3.7-Sonnet reaches 0.688 with revised prompts.Source: §6 · p. 6, Table 3 · p. 6

RAG Paradox

  • RAG raises cosine similarity for GPT-4o (0.379 to 0.524) and Claude-3.7-Sonnet (0.431 to 0.470), but lowers it for DeepSeek-V3 (0.407 to 0.379).Varying the number of retrieved documents likewise improves GPT-4o, degrades Claude-3.7-Sonnet, and leaves DeepSeek-V3 unstable.Source: §6 · pp. 6–7, Table 3 · p. 6

Prompt Ablation: Structure vs. Style

  • Revised and structure-only prompts improve specificity and actionability over normal and style-only prompts, with structure-only strongest on these direct quality dimensions.RAG variants sometimes improve actionability but show less stable specificity and tend to reduce faithfulness.Source: §6 · p. 6, Table 4 · p. 7

Human Evaluation

  • In human evaluation, human reviews perform best and enhanced prompting improves on standard prompts; RAG helps GPT-4o but harms Claude-3.7-Sonnet and DeepSeek-V3.The evaluation covers 160 reviews across four methods and 40 randomly selected papers.Source: §6 · p. 7, Table 5 · p. 7

Evidence-Grounded Correctness Analysis

  • Revised prompts reduce unsupported and nonverifiable claims; in pairwise comparisons, revised reviews are often most specific and more constructive than normal reviews, while human reviews remain most faithful.Source: §6 · p. 7

7 Conclusion

PeerCheck compares human and LLM reviews, then uses revised prompting to improve generated reviews. Results support prompt and model-specific strategies rather than simply adding more input or data.

Conclusion

  • PeerCheck compares human-written and LLM-generated reviews, then applies Chain-of-Thought prompting to revise review generation.Source: §7 · p. 7
  • Revised prompts reduce LLM-generated review detection rates by 40.3% and increase similarity to human-written reviews by 26.6%.Source: §7 · p. 7
  • Claude-3.7-Sonnet creates the most human-like reviews, GPT-4o best describes strengths, and DeepSeek-V3 improves most with revised prompts.The authors conclude that review improvement calls for planned prompts and model-specific strategies, not simply more input length or data volume.Source: §7 · p. 7

Limitations

The study’s findings are bounded by its model, corpus, persona, and ethical scope. The authors identify limits to generalization and raise transparency concerns about reduced review detectability.

Limitations

  • The evaluation uses three LLMs and focuses on post-2023 machine-learning papers, so generalization to other fields or earlier periods is uncertain.Source: Limitations · p. 7
  • RAG relies on existing human-review data, which may limit its effectiveness in emerging research areas.Source: Limitations · p. 7
  • Role-sensitivity testing covers only five academic personas, and reduced detection rates raise ethical questions about transparency and recognizability.Source: Limitations · pp. 7–8

Ethical Consideration

These supplementary sections document the study’s ethical boundaries, prompt variants, dataset construction, and review-comparison metrics. They specify how reviews are generated and assessed, while defining human-level quality narrowly as similarity to historical reviews.

Ethical Consideration

  • PeerCheck is intended as an assistive analytical tool, and similarity to historical reviews does not capture every dimension of review quality.The paper notes potential linguistic and cultural bias and says lexical and semantic similarity should not be treated as a normative definition of quality.Source: Ethical Consideration · p. 8

A.1 Evaluation Prompt Sample; A.2 Role-play Prompt Samples

  • Evaluation prompts request strengths, weaknesses, constructive feedback, and a score, while role-play prompts vary reviewer seniority and professional background.The sample evaluation prompt asks for critical assessment and warns against uniformly high scores; personas include undergraduate, PhD student, industry expert, and professor.Source: §A.1 · p. 11, §A.2 · pp. 11–12

A.3 Revised Prompt Samples

  • Revised prompts prescribe a polished, human-like review format, with concrete comparisons, specific weaknesses, and actionable questions.Some variants also require first-person phrasing, light hedging, and references to figures, equations, or hyperparameters.Source: §A.3 · p. 12

A.4 Prompt Samples for Prompt Ablation

  • Prompt-ablation variants separate stylistic instructions from structure and substantive grounding requirements.The style-only prompt encourages natural phrasing without forced citations or numeric comparisons; structure-only instructions require paper-specific pointers and actionable feedback.Source: §A.4 · p. 13

B Datasets Details

  • The corpus includes 1,920 papers for review generation and analysis and 480 validation papers, drawn from ICLR 2024, ICLR 2025, and NeurIPS 2024.Human-written reviews were collected from OpenReview for comparison with LLM-generated reviews.Source: B Datasets Details · p. 13

C Details for LLM Detection Metrics

  • The appendix catalogs detection approaches for distinguishing human from LLM text and similarity measures for comparing their language.The listed approaches include a RoBERTa classifier and measures such as BLEU and ROUGE.Source: C Details for LLM Detection Metrics · p. 13

E More Results for Similarity

Revised prompts generally improve similarity to human reviews, but retrieval augmentation has model- and review-section-dependent effects. Questions are the least human-similar section, and some models worsen with RAG.

Questions Section Similarity Analysis

  • Questions show the lowest human similarity; with RAG, DeepSeek-V3 falls from 0.254 to 0.203, while model responses to retrieval differ.GPT-4o improves most in the questions analysis, whereas Claude-3.7-Sonnet improves slightly with RAG and more with further modifications.Source: E More Results for Similarity · pp. 14–15

Weaknesses Section Similarity Analysis

  • Revised prompts improve weakness-section similarity across all three LLMs, with Claude-3.7-Sonnet reaching 0.492 in the revised-prompt keywords condition on cosine similarity.Claude-3.7-Sonnet reaches 0.429 on cosine similarity with RAG and revised prompts; the text notes that RAG only partly resolves limitations in critical appraisal.Source: E More Results for Similarity · p. 14, Table 8 · p. 15

Strengths Section Similarity Analysis

  • For strengths, revised prompts with keywords raise Claude-3.7-Sonnet similarity to 0.478, while RAG effects vary across models.GPT-4o improves with RAG, but Claude-3.7-Sonnet declines substantially and DeepSeek-V3 improves only slightly in human similarity.Source: E More Results for Similarity · p. 14, Table 7 · p. 15

F Ablation Study for Evidence-Grounded Correctness Analysis

Across these analyses, revised prompts improve claim faithfulness and lexical diversity, while RAG effects vary by model and retrieval volume. Human-written reviews still score highest overall and are most often judged faithful.

F Ablation Study for Evidence-Grounded Correctness Analysis

  • 0.209 to 0.174 was the unsupported-claim rate change under revised versus normal prompts without RAG; the not-verifiable rate fell from 0.311 to 0.198.Keyword and sample augmentations further lowered these rates, while improvements under RAG were less stable.Source: F Ablation Study for Evidence-Grounded… · p. 15, Table 10 · p. 16

G Ablation Study for RAG Paradox

  • 0.426 at k=1 to 0.594 at k=7 was GPT-4o’s fourth-metric change under normal prompts, while Claude-3.7-Sonnet’s second metric fell from 0.353 at k=1 to 0.166 at k=5.DeepSeek-V3’s second metric under normal prompts fluctuated from 0.289 at k=1 to 0.154 at k=7, indicating no consistent trend.Source: G Ablation Study for RAG Paradox · p. 16, Table 11 · p. 17

H More Details for Linguistic Metric

I.1 Human Evaluation Methodology

  • Three expert annotators blindly assessed 160 reviews from 40 papers using a structured rubric focused on insightfulness, constructiveness, and helpfulness.The study also describes a five-dimension 1–10 evaluation scale.Source: §I.1 · pp. 16–17

I.2 Expanded Human Evaluation With Multidimensional Criteria

  • 42.50% of reviews selected human-written were judged most faithful; revised reviews were most often selected as most specific (45.21%) and were chosen more often as most constructive (28.51%) than normal reviews (11.25%).Revised-with-RAG reviews performed worse than revised reviews across all three dimensions.Source: §I.2 · p. 17, Table 13 · p. 17

I.3 Detailed Results

  • 7.12 was the average for human-written reviews, versus 5.37–6.68 for LLM reviews; high-quality responses were 66.95% for human reviews and 60.33% for revised LLM reviews.High-quality response rates were 42.50% for RAG-enhanced reviews and 31.67% for normal LLM reviews; inter-rater agreement was κ = 0.707 (85.00%).Source: §I.3 · p. 17

I.4 Case Study

The case study contrasts human reviews’ close scrutiny of scholarship with three LLM review styles. Revised reviews are more critical, while RAG-enhanced reviews emphasize practical and implementation concerns.

I.4 Case Study

  • Human reviews scrutinize factual accuracy, citation completeness, originality, and whether claimed contributions are novel.Their example is described as strict and questioning-driven, extending from experiments to formatting.Source: §I.4 · p. 18, Figure 11 · p. 18
  • Normal LLM reviews offer standardized, procedural feedback on experiment clarity, reproducibility, and usefulness, but tend toward generic suggestions and limited criticism.The example review is described as easy on authors’ self-statements and less critical of technical specifics.Source: §I.4 · p. 18, Figure 12 · p. 19
  • Revised LLM reviews are more detailed and critical, identifying limited experimental scope, missing failure-case analysis, and incomplete privacy accounting.They also ask authors to demonstrate practical usability, reproducibility, and robustness of evaluation metrics.Source: §I.4 · p. 18, Figure 13 · p. 20
  • RAG-enhanced reviews emphasize implementation details, computational requirements, ablations, scalability, and generalizability.Across the examples, LLM reviews are constructive but still fall short of human reviews in academic rigor and literature attribution.Source: §I.4 · pp. 18–19, Figure 14 · p. 21
Loading 2606.20897v1…