PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM-REVal: Can We Trust LLM Reviewers Yet?

Rui Li, Jia-Chen Gu, Po-Nien Kung, Heming Xia, Junfeng liu, Xiangwen Kong, Zhifang Sui, Nanyun Peng

arXiv:2510.12367v1cs.CLcs.AI

TL;DR

As LLMs enter both research and peer review, the fairness implications of their coupled roles remain underexplored. LLM-REVal simulates that cycle and finds systematic mismatches between LLM reviewers and human judgments, including higher ratings for LLM-authored papers and undervaluation of critical human work.

  • Problem

    How LLMs’ deep integration into both research and peer review affects scholarly fairness remains underexplored.

    Source: §1 · pp. 1–2

  • Method

    LLM-REVal simulates research and review agents through submission and revision rounds, then compares LLM-review patterns with human judgments.

    Source: §1 · p. 2

  • Results

    LLM reviewers favor LLM-authored papers and LLM-style writing while persistently underrating some critical human-authored work, misaligning with human judgments.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings indicate risks to fairness for human authors when LLMs review research, particularly when evaluating LLM-generated work.

    Source: §8 · p. 10

    High token costs limited the initial human-authored sample to 100 papers.

    Source: §4.2 · p. 5

Abstract

from arXiv · show

The rapid advancement of large language models (LLMs) has inspired researchers to integrate them extensively into the academic workflow, potentially reshaping how research is practiced and reviewed. While previous studies highlight the potential of LLMs in supporting research and peer review, their dual roles in the academic workflow and the complex interplay between research and review bring new risks that remain largely underexplored. In this study, we focus on how the deep integration of LLMs into both peer-review and research processes may influence scholarly fairness, examining the potential risks of using LLMs as reviewers by simulation. This simulation incorporates a research agent, which generates papers and revises, alongside a review agent, which assesses the submissions. Based on the simulation results, we conduct human annotations and identify pronounced misalignment between LLM-based reviews and human judgments: (1) LLM reviewers systematically inflate scores for LLM-authored papers, assigning them markedly higher scores than human-authored ones; (2) LLM reviewers persistently underrate human-authored papers with critical statements (e.g., risk, fairness), even after multiple revisions. Our analysis reveals that these stem from two primary biases in LLM reviewers: a linguistic feature bias favoring LLM-generated writing styles, and an aversion toward critical statements. These results highlight the risks and equity concerns posed to human authors and academic research if LLMs are deployed in the peer review cycle without adequate caution. On the other hand, revisions guided by LLM reviews yield quality gains in both LLM-based and human evaluations, illustrating the potential of the LLMs-as-reviewers for early-stage researchers and enhancing low-quality papers.

1 Introduction

As LLMs take on roles in both research and peer review, the effects of this feedback loop on scholarly fairness remain underexplored. LLM-REVal simulates that cycle and finds systematic reviewer biases that misalign with human judgments.

1 Introduction

  • The paper situates its study amid growing interest in LLM support for research and peer review, including reports of clearer, more actionable reviews and high-quality meta-review summaries.The unresolved issue is how the dual roles of LLMs in research and review interact in the academic process.Source: §1 · pp. 1–2
  • LLM-REVal simulates research and review agents across an initial submission round and multiple rounds revising low-scoring papers with LLM review guidance.The review agent models assessment, rebuttals, reassessment, meta-review, and final decisions.Source: §1 · p. 2, Figure 1 · p. 2
  • LLM reviewers give LLM-authored papers higher scores than human-authored papers, while human reviewers do not share that preference and judge some persistently low-scored human papers valuable.The simulation also finds score gains after revisions for both authorship groups.Source: §1 · p. 2
  • LLM reviewers favor LLM-style linguistic features and exhibit aversion to critical statements, tending to inflate scores for the former and undervalue work containing the latter.The cited critical topics include risk and fairness.Source: §1 · p. 2
  • The authors warn that irresponsible deployment of LLM reviewers may pose fairness risks to human authors and erode trust in peer review.They call for safeguards as LLMs become integrated across the research lifecycle.Source: §1 · p. 3

2 Related Work

Prior work documents both the promise of LLMs in research and peer review and limitations in reviewer behavior. The paper addresses a gap: much prior reviewing research treated that stage separately from the broader scholarly lifecycle.

AI-assisted Review

  • Prior peer-review studies report useful paper feedback, improved review clarity and actionability, and strong multi-perspective meta-review summaries (Liang et al., 2023; Thakkar et al., 2025).Reviewer agents include static systems such as Reviewer2 (Gao et al., 2024b), MARG (D’Arcy et al., 2024), and DeepReview (Zhu et al., 2025b), as well as dynamic systems such as Agentreview (Jin et al., 2024) and ReviewMT (Tan et al., 2024).Source: §2 · p. 3
  • Prior work reports reviewer risks including overemphasis on stated limitations, inflated scores for submissions with limited substance, low sensitivity to ethical or technical issues, and vulnerability to perturbations and hidden prompts.The authors note that these studies largely examine reviewing in isolation from other scholarly stages.Source: §2 · p. 3

AI-assisted Research

  • Research applications span idea generation, citation, experiment design and analysis, and increasingly autonomous end-to-end pipelines, with some generated papers reaching ICLR workshops and ACL.The passage also describes reported strengths and limitations in idea novelty and scholarly information retrieval.Source: §2 · p. 3

3 Simulation Construction

The simulation couples a research agent that generates and revises manuscripts with a review agent that evaluates them. Its repeated review-and-revision cycle enables analysis of review dynamics across human- and LLM-authored submissions.

3.1 Research Agent

  • The research agent retrieves literature, generates and ranks ideas, designs studies, predicts results, drafts papers, and can revise manuscripts.The simulation predicts results rather than iterating experiments, citing limits in LLM coding capability and the time needed for execution.Source: §3.1 · pp. 3–4
  • The research agent uses retrieved literature to guide ideas, experimental design, references, and background, then drafts papers sequentially within an ICLR template.Its writing workflow combines rule-based components with LLM generation and verifies generated references against original works.Source: §3.1 · p. 4

3.2 Review Agent

  • The review agent models a five-stage peer-review workflow, from reviewer assessments and author–reviewer discussion through reassessment, meta-review, and final decision.Three reviewers score papers on a 1–10 scale; the average score across stages determines acceptance, using 6 as the threshold.Source: §3.2 · p. 4

3 Simulation Construction

  • The simulation repeats research and review stages across one initial round and multiple revise-review rounds, using average scores and detailed reviews to guide progression and revisions.The research agent supplies papers and revisions, while the review agent evaluates the submissions.Source: §3 · p. 3, §3.2 · p. 4, §3.3 · p. 5

3.3 Simulation Workflow

  • Round 1 compares human-authored submissions with research-agent papers generated using keywords extracted from human papers, with scores determining acceptance or revision.Only rejected papers enter subsequent rounds; both authorship groups are revised using their LLM reviews, for up to six rounds or until all previously rejected papers are accepted.Source: §3.3 · p. 5

4 Simulation Statistics and Construction

The study uses selected DeepSeek models for research and review, samples human-authored ICLR papers across ten topics, and validates the review agent against human judgments.

4.1 Models

  • DeepSeek-R1 generates research ideas and reviews papers, while DeepSeek-V3 handles writing and revisions; additional review experiments use GPT-4o, Qwen3, and Gemini-2.5.The model choices reflect preliminary assessments of reasoning, generation quality, hallucination rate, inference speed, and cost.Source: §4.1 · p. 5

4.2 Human Paper Collection

  • The human-paper collection includes 100 ICLR submissions sampled across ten research topics; high token costs and preliminary performance observations limited the initial human-authored set to 100.Generating a paper required approximately 500k input tokens, and reviewing each paper consumed about 200k input tokens.Source: §4.2 · p. 5

4.3 Validation of the Review Framework

  • 73.7% accuracy was achieved for acceptance prediction, and LLM review scores correlated positively with human scores (Pearson r = 0.5046, p = 8.61 × 10−8).The acceptance prediction used an average review score of 6 as its threshold; Figure 2 shows LLM-score distributions across human-score ranges.Source: §4.3 · p. 6, Figure 2 · p. 6

5 What Salient Patterns Do LLM Reviewers Emerge?

In Round 1, LLM-authored papers received higher review scores across most topics and prevailed in 66% of same-keyword comparisons, versus 26% for human-authored papers. In later rounds, revisions raised scores for both authorship groups, but some human-authored papers remained unaccepted through Round 6.

5.1 Research-Review (Round 1)

  • LLM-authored submissions won 66% of same-keyword comparisons versus 26% for human-authored papers, with 8% tied.LLM-authored papers also had a 78% acceptance rate versus 49% for human-authored submissions.Source: §5.1 · pp. 6–7, Figure 3 · p. 6, Table 1 · p. 6

5.2 Revise-Review (Round 2 to 6)

  • Average scores rose significantly after revision: LLM-authored papers increased from 5.79 to 6.09 (+0.30), and human-authored papers from 5.66 to 6.02 (+0.36).Both improvements were significant by t-test (p ≪0.05).Source: §5.2 · p. 7
  • By Round 3 all initially submitted LLM-authored papers had been accepted, while 5% of human-authored papers remained unaccepted by Round 6.The authors interpret this pattern as suggesting that some human-authored papers faced systematic disadvantage under LLM review.Source: §5.2 · p. 7
  • All patterns can be consistently observed across different reviewer backbones; the cited passage says detailed results are provided in Appendix A.6.Source: §5.2 · p. 7

6 Do The LLM-Review Patterns Align with Human Judgments?

Human annotations found that LLM-review score advantages did not align with human judgments, and that revisions addressed some review-identified issues. Real ICLR reviews also indicated that papers persistently scored low in simulation could be valued by expert human reviewers.

Human check for LLM-authored Superiority

  • Among 15 matched high-disparity pairs, annotators chose human-authored papers as superior in 56.7% of cases and LLM-authored papers in 33.3%.The pairs shared keywords and had average LLM and human review scores of 6.5 and 5.68, respectively.Source: §6 · p. 7
  • The LLM-authored superiority pattern persisted across diverse reviewer backbones.The authors describe the preference as a general phenomenon across diverse LLM architectures.Source: §6 · p. 8

Human check for Revision Boost

  • Of 174 text-editable issues identified in 22 LLM-authored paper–revision pairs, 81 (46.55%) showed improvement in revision.The 56 issues excluded from this comparison could not have been resolved through textual edits.Source: §6 · p. 8

Human check for Inevitable Rejection

  • Real ICLR reviews gave persistently low-scored human-authored papers about 5/10 on average, and one had been accepted to ICLR 2024.The authors conclude that research valued by expert human reviewers could nonetheless be systematically undervalued by LLM reviewers.Source: §6 · p. 8

7 What are the sources of bias in LLM reviewers?

The analysis identifies linguistic features associated with LLM-authored papers and examines how those features relate to review scores. It also finds that critical-topic framing is associated with lower LLM review scores for human-authored papers.

7.1 Linguistic Feature

  • LLM-authored papers differ from human-authored papers in length, lexical diversity, and complexity, features examined against review scores in Figure 5.Figure 5 plots review scores against linguistic-feature metrics.Source: §7.1 · p. 8, Figure 5 · p. 8
  • LLM-authored papers are more concise, with shorter overall paper, sentence, and paragraph lengths than human-authored papers.Source: §7.1 · p. 8
  • 1-gram diversity was 0.4321 for LLM-authored papers versus 0.2598 for human-authored papers, nearly twice as high.Source: §7.1 · p. 9
  • LLM-authored papers had higher FKG scores and dependency distances but lower subclause ratios than human-authored papers.The authors interpret this as longer, more morphologically complex words and longer spans between syntactically related words, alongside shallower hierarchical structure.Source: §7.1 · p. 9
  • After 40% LLM polishing, human papers' average score rose from 5.69 to 5.94, while their linguistic statistics shifted toward LLM-authored papers except for syntactic complexity.Source: §7.1 · p. 9

7.2 Critical Statement

  • For human-authored papers, negative keywords were associated with lower review scores, while LLM-authored papers' scores increased as negative-keyword counts rose.The authors suggest negative framing of critical topics may exacerbate review bias; positive framing of similar topics in LLM-authored papers tended to receive higher scores.Source: §7.2 · p. 9

7 What are the sources of bias in LLM reviewers?

  • The authors warn that style-linked score inflation and the systematic undervaluation of critical topics raise fairness concerns for LLM reviewers.They also state that LLM-review-guided revisions yield quality gains in LLM-based and human evaluations.Source: §7.2 · p. 9

8 Conclusion

The paper concludes that LLM reviewers exhibit systematic biases and misalign with human judgments, raising fairness concerns for their use in scholarly workflows. Its ethics statement describes an offline simulation using public papers, and it reports steps taken to support reproducibility.

8 Conclusion

  • The simulation found LLM reviewers overrate LLM-authored papers and undervalue critical human-authored work, producing marked misalignment with human judgments.The conclusion attributes these biases to linguistic-feature preferences and framing effects in critical discussion.Source: §8 · p. 10

Ethics Statement

  • The study used publicly available human-authored papers and conducted evaluations in a controlled offline setting that did not influence real peer-review decisions.The authors advocate human oversight and rigorous ethical guidelines for real-world deployment.Source: Ethics Statement · p. 10

Reproducibility Statement

  • The authors report that prompts, model versions, pipeline steps, statistical methods, and feature-extraction procedures are documented to support reproducibility.The human-paper dataset was sampled from public ICLR submissions, with corresponding LaTeX sources obtained from arXiv.Source: Reproducibility Statement · p. 10

A.1 Prompt

The author-agent prompts draw on Si et al. (2025) for idea generation and specify an abstract-generation prompt for paper generation; review-related prompts follow Jin et al. (2024).

A.1.1 Prompt for Author Agent

  • Idea-generation prompts are consistent with Si et al. (2025).Source: §A.1.1 · p. 15
  • The paper-generation prompt asks the author agent to produce an abstract from supplied paper information.Source: §A.1.1 · p. 15
  • The requested abstract should explain the research goal, its relevance, and why the problem is difficult.Source: §A.1.1 · p. 15
  • The abstract prompt asks authors to describe their solution and how experiments or results verify it.Source: §A.1.1 · p. 15
  • The prompt requires a smoothly motivated, continuous paragraph in ICLR-style academic writing.Source: §A.1.1 · p. 15
  • The prompt specifies beginabstract and endabstract wrappers and standard LaTeX content, with LaTeX wrapped as instructed.Source: §A.1.1 · p. 15

A.1.2 Prompt for Review Agent

A.2 Example of LLM-authored Paper

The appendix points to an example of an LLM-authored paper in Figure 7.

A.2 Example of LLM-authored Paper

  • Figure 7 presents an example of an LLM-authored paper.Source: §A.2 · p. 19

A.3 Example of LLM Review

Reviewers describe a tabular-data fairness study as relevant but largely incremental, citing gaps in analysis, methodological detail, and mitigation strategies. After rebuttals, they still judged that planned revisions did not resolve key limitations.

A.3 Example of LLM Review

  • Reviewers identify missing methodological detail on few-shot selection, label flipping, and fine-tuning settings as obstacles to replication and evaluation.The authors said they would expand those descriptions in a revision.Source: §A.3 · pp. 19, 23
  • Reviewers describe the dataset and protected-attribute choices as conventional and the comparison with traditional models as too superficial to establish generalizability.The authors said they would add datasets and deeper comparative analysis in future work.Source: §A.3 · pp. 21, 22, 24
  • The reviews recognize fairness in tabular prediction as an underexplored topic, but question whether the findings and methods offer enough novelty.Reviewers note that the work adapts existing mitigation strategies to this domain but largely confirms known biases.Source: §A.3 · pp. 21–23
  • Reviewers say the analysis does not sufficiently explain bias mechanisms, dataset-specific differences, or why mitigation strategies vary in effectiveness.They also flag limited theoretical grounding and insufficient discussion of societal or practical implications.Source: §A.3 · pp. 21–23
  • Reviewers say the paper tests no novel mitigation strategy and offers limited actionable guidance for real-world applications.The authors described broader mitigation strategies and practical implications as future additions.Source: §A.3 · pp. 21, 23, 24
  • Reviewers maintained or lowered their ratings after rebuttals, judging that planned revisions had not closed the current paper’s gaps.One reviewer maintained a score of 6; other reviews said the gaps remained insufficiently addressed.Source: §A.3 · p. 26

A.4 Negative Keywords

The appendix describes human annotation procedures and additional reviewer-model simulations. Across those models, LLM-authored papers scored higher; revision gains and persistent rejection patterns varied by reviewer, while the appendix also reports linguistic-feature statistics.

Negative Keywords

  • Table 3 lists the negative keywords used for bias-source analysis.The study used keyword detection to identify papers and then examined sentiment polarity in their abstracts.Source: Table 3 · p. 27

A.5.1 Human Annotation for LLM-authored Superiority

  • 15 human-versus-LLM paper pairs with the largest score differences averaged about 0.8 points, and annotators judged papers without knowing their authorship.Thirteen graduate-level or higher annotators assessed the pairs; each pair was independently checked by two annotators, with ties allowed.Source: §A.5.1 · pp. 27–28

A.5.2 Human Annotation for Revision Boost

  • Human annotators judged 81 of 174 issues, or 46.55%, improved after revision, excluding 56 issues that required substantial practical work.The review-comment analysis covered 22 original LLM-authored papers and their revisions.Source: §A.5.2 · p. 28

A.6.1 Additional LLMs as Reviewers: Simulation Result

  • Across all additional reviewer models, LLM-authored papers received higher average scores than human-authored papers.Table 4 reports results for three topics and multiple reviewer models.Source: Table 4 · p. 29, §A.6.1 · p. 29
  • Among papers initially scored low by DeepSeek-R1, revisions significantly improved scores with DeepSeek-R1, GPT-4o, and Qwen3, but not Gemini.Gemini had not assigned these papers low scores initially; Table 5 compares original and revised scores.Source: §A.6.1 · p. 29, Table 5 · p. 29
  • 5% of human-authored papers remained unaccepted after five revision cycles across all reviewer models.The scores were tracked from original versions through v5; the authors describe this as a potential systematic bias.Source: §A.6.1 · p. 29, Table 6 · p. 29

A.6.2 Additional Statistics

  • Figure 9 plots average length, lexical-diversity, and complexity features across human-authored, polished, and pure LLM-authored documents.Its x-axis distinguishes document conditions and its y-axis gives average feature values; Tables 7 and 8 are titled as linguistic-statistics and feature-correlation tables.Source: Figure 9 · p. 30, Table 7 · p. 30, Table 8 · p. 31

Related papers

Loading 2510.12367v1…