Source-linked AI summary
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Peng Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Ranran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Jiayang Cheng, Zhaowei Wang, Ying Su, Raj Sanjay Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip S. Yu, Wenpeng Yin
TL;DR
Researchers face growing workloads in reading, writing, and reviewing, motivating questions about LLM assistance in expert peer review. This work builds ReviewCritique to compare human and LLM reviews and evaluate LLMs as meta-reviewers. LLM-generated reviews often contain deficient, paper-unspecific segments, while even top-tier LLMs struggle to assess review deficiencies like human experts.
Problem
The paper asks whether LLMs can effectively assist NLP paper reviewing and meta-reviewing as review workloads increase and expert judgment remains necessary.
Method
The authors create ReviewCritique, pairing initial NLP submissions with human and LLM-generated reviews and expert sentence-level deficiency annotations with explanations.
Results
LLM-generated reviews often contain more deficient, paper-unspecific segments lacking diversity and constructive feedback, while even top-tier LLMs struggle to assess individual reviews like human experts.
Takeaways & Limitations
The findings indicate that LLMs are currently insufficient for automating peer review tasks requiring expert judgment and nuanced understanding.
Takeaways & Limitations
The evaluation focuses on textual information from NLP papers and the pre-rebuttal phase, excluding figures, tables, other visual elements, and other research domains.
Abstract
from arXiv · showhide
This work is motivated by two key trends. On one hand, large language models (LLMs) have shown remarkable versatility in various generative tasks such as writing, drawing, and question answering, significantly reducing the time required for many routine tasks. On the other hand, researchers, whose work is not only time-consuming but also highly expertise-demanding, face increasing challenges as they have to spend more time reading, writing, and reviewing papers. This raises the question: how can LLMs potentially assist researchers in alleviating their heavy workload? This study focuses on the topic of LLMs assist NLP Researchers, particularly examining the effectiveness of LLM in assisting paper (meta-)reviewing and its recognizability. To address this, we constructed the ReviewCritique dataset, which includes two types of information: (i) NLP papers (initial submissions rather than camera-ready) with both human-written and LLM-generated reviews, and (ii) each review comes with "deficiency" labels and corresponding explanations for individual segments, annotated by experts. Using ReviewCritique, this study explores two threads of research questions: (i) "LLMs as Reviewers", how do reviews generated by LLMs compare with those written by humans in terms of quality and distinguishability? (ii) "LLMs as Metareviewers", how effectively can LLMs identify potential issues, such as Deficient or unprofessional review segments, within individual paper reviews? To our knowledge, this is the first work to provide such a comprehensive analysis.
1 Introduction
This work examines whether LLMs can assist NLP paper reviewing and meta-reviewing, using expert-annotated comparisons of human and LLM-generated reviews. It finds substantial deficiencies in LLM-generated reviews and difficulty matching human expertise in evaluating reviews.
- The study asks whether LLM-generated reviews differ from human reviews and whether LLMs can identify deficient segments in human-written reviews.
- ReviewCritique pairs initial NLP submissions with human-written and LLM-generated reviews, plus expert sentence-level deficiency and professionalism annotations with explanations.
- LLMs as Reviewers: LLM-generated reviews contain more deficient segments and often lack paper specificity, diversity, and constructive feedback.
- LLMs as Metareviewers: Even top-tier LLMs struggle to assess individual reviews like human experts when meta-reviewing.
- The work contributes a dataset, the first sentence-level quantitative comparison of human and LLM reviews, and analysis of LLMs as reviewers and meta-reviewers.
2 Related Work
Prior work applies AI to generate and evaluate peer reviews, while existing datasets generally lack expert annotations or comprehensive, expertise-demanding review assessment. ReviewCritique addresses these gaps through fine-grained comparison and benchmarking of responsible meta-reviewing.
- Prior research studies automated score prediction, review and meta-review generation, and NLP-based evaluation of human review quality.
- Existing review datasets often contain papers and reviews without expert annotations, while others target specific facets such as argumentation, politeness, uncertainty, or contradictions.
- ReviewCritique is presented as the first dataset to benchmark LLMs as responsible meta-reviewers, requiring highly experienced annotators to assess reviews in context.
- Unlike previous work, ReviewCritique quantitatively compares human and LLM-generated reviews at the sentence level and introduces a metric for LLM review diversity.
3 ReviewCritique Curation
ReviewCritique combines initial NLP submissions with human and LLM-generated reviews, expert sentence-level deficiency annotations, and quality controls to support granular comparison and meta-reviewer benchmarking.
- Dataset construction: ReviewCritique contains 100 NLP submissions from ICLR and NeurIPS, complete individual reviews, meta-reviews, and rebuttals, with papers balanced between accepted and rejected outcomes.The collected reviews comprise 3–5 individual reviews per submission and use the latest pre-deadline submission versions.
- Dataset construction: The dataset compares human-written reviews with LLM-generated reviews using a 20-paper subset selected for feasible annotation and balanced acceptance outcomes.Three closed-source LLMs were used to generate comparison reviews.
- Data annotation: Forty experienced NLP annotators labeled review sentences for factual errors, misinterpretations, unsupported downsides, lack of constructive feedback, and subjective or offensive judgments.The team included researchers with multiple first-authored top-tier publications, including Ph.D. holders, faculty members, and former area chairs.
- Data annotation: Annotations were based on the original submission before rebuttal, with annotators reading the paper and review materials while separate annotators assessed LLM-generated reviews without being told their origin.The authors acknowledge that concealing submission information cannot guarantee complete annotator unawareness.
- Data annotation: Two annotators independently reviewed each paper’s reviews, disagreements were resolved by a senior area-chair-level expert, and regular meetings supported a six-month annotation process.The quality-control procedure included removing or rewriting explanations judged unconvincing.
- Data statistics: LLM-generated reviews contained more deficient instances than human reviews at both review and segment levels, with rates of 100% versus 71.57% and 13.97% versus 6.27%, respectively.For accepted submissions, LLM-generated reviews had more deficient segments than for rejected submissions, unlike human-written reviews; the authors relate this pattern to LLMs’ superficial, non-specific criticism.
4 Experiments
Experiments compare LLM-generated and human reviews, then test LLMs as metareviewers identifying deficient segments and explaining their judgments. LLM reviews are less specific and constructive, while metareviewer performance remains limited even for closed-source models.
- LLMs as Reviewers: LLM-generated strengths often rephrase authors’ claims, with 53.2% copied from submissions, and human experts questioned 34.5% of accepted-paper and 51.9% of rejected-paper strength segments.This indicates limited critical verification of claimed contributions.
- LLMs as Reviewers: LLM-generated reviews contain more deficient segments than human reviews and often produce paper-unspecific, superficial feedback.They particularly struggle to provide constructive, scope-aware weaknesses and accurate writing assessments.
- LLMs as Reviewers: Human reviews are more diverse than LLM reviews: ITF-IDF scores 6.04 for humans, 4.09 for Claude Opus, 3.82 for Gemini, and 3.34 for GPT-4.GPT-4 shows the lowest scores, while LLM diversity drops sharply in the Clarity section.
- LLMs as Reviewers: Inter-LLM BERTScores range from 70.33 to 71.17, versus 59.15 between human reviews, suggesting multiple LLMs add less perspective diversity.The higher similarity among LLM reviews indicates complementary viewpoints are limited.
- LLMs as Metareviewers: Closed-source LLMs generally outperform open-source models in deficient-segment F1, but consistently higher recall than precision indicates frequent false deficient judgments.Even the stronger closed-source systems retain relatively low F1 scores, underscoring the difficulty of expertise-intensive metareviewing.
- LLMs as Metareviewers: LLMs identify some deficient segments but struggle to explain their reasoning, with Claude Opus aligning best with expert explanations and open-source models generally scoring lower.Explanation scores remain relatively low across models.
5 Conclusion
The study examines LLMs as reviewers and meta-reviewers and finds persistent deficiencies in both roles, limiting their current usefulness for automating peer review.
- LLMs often generate Deficient, paper-unspecific reviews that lack diversity and constructive feedback, while even state-of-the-art models struggle to assess review deficiencies.
Limitations
The evaluation is limited to textual NLP submissions and the pre-rebuttal review stage, leaving visual information, other domains, and later review turns unexamined.
- ReviewCritique excludes figures, tables, and other visual elements, so it does not assess LLM review capabilities over complete multimodal submissions.
- The dataset covers only NLP papers, limiting direct evidence about generalization to other research domains.
- The study evaluates original submissions before rebuttals, excluding author responses and post-rebuttal reviews from the analyzed process.
Ethical Considerations
The authors discuss risks of LLM-assisted reviewing and emphasize that current systems should not replace human reviewers for expert, nuanced judgments.
- Potential risks include bias, lack of accountability, and undermining the integrity of scientific evaluations.
- The study does not advocate LLM use for paper reviewing because current systems remain insufficient for expert judgment and nuanced understanding.
A Experiment Details
The experiments compare closed- and open-source LLMs using similarity-based evaluation components, with prompt construction adjusted for model context limits.
- Similarity evaluation: BERTScore uses DeBERTa-xlarge-MNLI for explanation evaluation and BART-large-MNLI for inter-LLM complementarity because full reviews can exceed 512 tokens.
- Similarity evaluation: SentenceBERT with all-MiniLM-L6-v2 computes ITF-IDF similarity, although any sentence-similarity model could be substituted.
- Model settings: Closed-source experiments use GPT-4, Gemini 1.5, and Claude 3, while open-source experiments use Llama3-8B, Llama3-70B, and Qwen2-72B.
- Prompt construction: For Llama3 models, the paper body is truncated to fit the 8k-token context limit while retaining instructions, title, and review text.
B Influence of Different Thresholds in ITF-IDF
Varying the similarity threshold t in ITF-IDF does not change the performance ranking on full reviews.
- The performance rank remains unchanged across different similarity-threshold values t.Table 6 reports ITF-IDF under multiple threshold settings for full reviews.
C Error Types Detected by LLMs
The study reports how often LLMs identify each error type in human-written reviews.
- The evaluation reports the number and percentage of human-review segments detected by each LLM for every error type.
D Deficient Segment Error Types
The paper defines review error categories and evaluates LLMs’ detection and explanation of deficient segments using multiple prompts and models.
- D Deficient Segment Error Types: Error types categorize deficient review segments, with annotation-team explanations and percentages reported for human-written and LLM-generated reviews.
- D Deficient Segment Error Types: LLM-generated explanations for correctly identified deficient segments are evaluated with ROUGE and BERTScore under two prompting strategies and their ensemble.The strategies are Labeling-All and Select-Deficient, with ensemble scores averaged across prompt-generated explanations.
- D Deficient Segment Error Types: The experiments use dedicated prompts for generating reviews and identifying deficient segments.
- D Deficient Segment Error Types: GPT-4, Claude, and Gemini are compared on identifying deficient segments across error types.