Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions

1 Introduction

• About 92% of the time, the attack raised a colluding reviewer’s similarity ranking from 101 to top-5 on NeurIPS 2023 data.The evaluated matching algorithm was SPECTER, used by top ML/AI conferences for reviewer assignments.Source: §1 · p. 2
• Ranking correlations of 0.62, 0.83, 0.92, and 0.93 across four settings indicate NeurIPS 2022 attack performance reflects NeurIPS 2023 performance.The attacker is assumed to have access to the previous year’s publicly available data, but not the current reviewer pool.Source: §1 · p. 2
• Selecting one reviewer-archive paper most similar to the target raised the reviewer from rank 101 to top-5 in 41% of cases, without modifying the abstract.Source: §1 · p. 2
• With multiple papers in a reviewer’s profile, maximum similarity was more vulnerable than mean similarity: top-5 success was 49% versus 32%.Source: §1 · p. 2
• In a human-subject experiment using automated attacks only, attacked abstracts drew more coherence or consistency complaints, but control abstracts also drew complaints.That control complaint rate may give attackers plausible deniability; the automated-only experiment provides an upper bound on detectability because attackers could otherwise inspect and iterate.Source: §1 · p. 2
• The work released code, datasets, and manipulation examples to raise awareness of a vulnerability with implications for peer-review fairness and integrity.The conference management platform implemented organizer safeguards, which were used by a major AI/ML conference; other venues expressed interest.Source: §1 · p. 2

2.1 Automated Reviewer Assignments

• Reviewer assignment optimizes similarity scores subject to reviewer and paper loads and conflicts; higher scores make assignment more likely, so the evaluation compares reviewers’ relative text-similarity rankings.The section isolates text similarity because bidding is already known to be vulnerable and can only increase manipulation success.Source: §2.1 · p. 3

2.2 Paper-Reviewer Text Similarity

• SPECTER computes text similarity from paper titles and abstracts, comparing a submission with papers in a reviewer’s archive and aggregating those similarities.The default archive contains the reviewer’s 10 most recent papers; max and average pooling are common, and reviewer-curated archives create an attack surface.Source: §2.2 · pp. 3–4

2.3 Attacker’s Threat Model

• The attacker is a colluding author-reviewer pair seeking to rank the reviewer in the top 1, 3, or 5 by text similarity to the target paper.The threat model assumes the pair can coordinate, while the current conference’s reviewer pool is unavailable to them.Source: §2.3 · p. 4
• The reviewer may curate an adversarial archive, while the author may modify the abstract; the study excludes title changes and constrains edits to remain coherent and consistent.The abstract changes are limited in sentences and keywords; up to three reviewer-related sentences are allowed in manually supervised cases, and up to 10 keywords.Source: §2.3 · p. 4

Text matching and collusion rings

• Prior PDF attacks concealed text from human readers while exploiting parsers, but lacked plausible deniability and risked authors’ reputations.Source: §3 · p. 5

Text-level attacks and collusion rings

• Earlier text-level attacks edited bibliographies, used synonyms or spelling errors, or inserted keywords to select or remove reviewers [EQM+23].Source: §3 · p. 5

1. Setting

• Compared with [EQM+23], this study evaluates 7,900 reviewers and 3,218 papers, targets original ranks 20–1001, drops the known-committee assumption, and adds archive curation.The earlier study used 165 reviewers and 32 papers and examined original ranks 6–10.Source: §3 · p. 5

2. Text matching model

• Unlike [EQM+23]’s LDA-based TPMS setting, this work studies a less interpretable neural embedding model, SPECTER, which is more widely used at ML/AI venues.Preliminary experiments found similar vulnerabilities in SPECTER version 2, but the paper focuses on the original model.Source: §3 · p. 5

3. Evaluation

• The evaluation focuses on assigning one paper to a colluding reviewer; text-only attacks reach about 92%, while earlier attacks without PDF tricks stayed below 70% even with aggressive modifications.The study also collected 116 human-subject samples, compared with 21 in the earlier work.Source: §3 · p. 6

Publicly released adversarial samples

• The authors publicly release more than 1,000 adversarial samples, contrasting with the three samples released by [CFB+20].Source: §3 · p. 6

4.1 Adversarial Archive Curation by the Colluding Reviewer

• With max pooling, retaining the reviewer’s single paper most similar to the target can bias assignment; the analysis uses a single-paper setting to indicate attack potential under max pooling and reviewer sub-selection under average pooling.The authors report that max pooling is more vulnerable than mean pooling in their comparison.Source: §4.1 · p. 7

4.2 Adversarial Abstract Modifications by the Colluding Author

• The author constructs a modified abstract to increase similarity to the reviewer’s adversarial archive while maintaining coherence and consistency.The attack uses LLMs in human-in-the-loop and fully automatic modes; human oversight is intended to avoid suspicious artifacts.Source: §4.2 · p. 7

The IncludeThemes operation

• IncludeThemes adds background or filler sentences related to archive-paper themes; in the Figure 2 example, the reviewer rises from 101st to most-similar at NeurIPS 2023.The operation exploits increased similarity from overlapping themes, even when those themes are only background information in the modified abstract.Source: §4.2 · p. 8, The IncludeThemes operation · p. 8

The InsertKeywords operation

• InsertKeywords adds archive-derived terms that can raise similarity across different meanings or parts of speech; in the example, the reviewer rises from 101st to third at NeurIPS 2023.Some terms are adapted to common English, and the operation may adjust surrounding text for coherence and grammatical correctness.Source: The InsertKeywords operation · pp. 8–9

5.1 Experiment Setup

• The experiments used NeurIPS 2022 as prior data and NeurIPS 2023 as the target, selecting N = 5, M = 2, K = 5 from tests on the prior-year dataset.The attack selects a reviewer’s most similar archived paper and modifies the submitted abstract; fully automatic experiments omit human supervision and early stopping for scalable evaluation.Source: §5.1 · pp. 10–11

5.2 Human-in-the-loop Mode Attack Results

• Human-in-the-loop attacks with early stopping improved reviewer rankings in most tested cases despite manual coherence and consistency checks.The researchers increased the IncludeThemes budget from 5 to 10 versions, allowed up to three archive-related sentences, and used K = 5 and M = 2; early stopping meant most abstracts received fewer than 10 added keywords.Source: §5.2 · p. 11, Table 1 · p. 12

5.3 Fully Automatic Mode Attack Results

• 93% top-5 success for natural rank 101 and 48% top-1 success for natural rank 1001 show that fully automatic attacks can work even for weakly matched colluders.Success was highest for natural rank 20; the authors report generally high success rates across the tested rankings.Source: §5.3 · p. 12, Table 2 · p. 12

5.4 Lower Limits on Reviewer Archive Length

• 30% of rank-101 pairs reached the top 3 through archive curation alone, with a mean manipulated rank of 24.59; requiring longer archives reduced attack success.The reduction comes with a trade-off: honest reviewers may want to update their profiles to reflect current research interests.Source: §5.4 · p. 13, Table 3 · p. 13, Figure 4 · p. 13

5.5 Maximum versus Average Similarity

• Without adversarial archive curation, abstract modification was more successful under maximum than average similarity aggregation, making maximum aggregation more susceptible to the attack.This comparison used rank-101 pairs with reviewers’ full archives retained.Source: §5.5 · p. 14, Table 4 · p. 14

5.6 Correlation of Attack Success between NeurIPS 2022 and NeurIPS 2023

• Manipulated rankings in NeurIPS 2022 and 2023 were strongly correlated, allowing attackers to use publicly available prior-year reviewer data to estimate success and tune modifications.The analysis covered 300 pairs at natural rank 101 and 100 pairs each at ranks 20, 501, and 1001.Source: Figure 5 · p. 14, §5.6 · p. 15

5.7 Human Subject Experiment on Detectability

• The human study compared benign and adversarial versions of real papers by assigning 58 experienced ML/AI reviewers two papers each and asking for ten-minute mini-reviews.Adversarial abstracts were automatically generated to place the colluding reviewer within the top 5; benign abstracts were LLM-paraphrased to reduce the human-written versus LLM-generated confound.Source: §5.7 · p. 15, §5.7.1 · p. 15, §5.7.4 · p. 16

5.7.5 Limitations

• The human-study design may overestimate detectability because abstracts were modified without human oversight, paper bodies were not edited, and mini-reviews focused attention on abstracts.Its prompts and budgets were also tuned on NeurIPS 2023 test data, and the study added 12 keywords without early stopping, versus 10 keywords in the other automatic experiments.Source: §5.7.5 · pp. 16–17, §5.7.2 · p. 16
• Authors enter titles and abstracts through web-interface text boxes, so entered abstracts may differ from PDFs that reviewers read more carefully; compensation may reduce some participants’ attention.Source: §5.7.5 · p. 17

5.7.6 Results

• Control abstracts were rated significantly more consistent than experimental abstracts, with no significant differences in expertise alignment or abstract coherence.The study collected 49 control and 67 experimental mini-reviews.Source: Table 5 · p. 17, §5.7.6 · p. 17
• Writing style was the most prevalent complaint in both groups, and manual checks found at least one original abstract matching each complaint type.Complaints that an abstract contained material absent from the paper appeared in both groups; all comments ranged from 7 to 189 words.Source: §5.7.6 · p. 17, Table 6 · p. 18
• No participant identified malicious intent; 6% of experimental-group responses related to IncludeThemes, 16% to InsertKeywords, and 82% to neither operation.Some responses may relate to both operations, so the percentages do not sum to 100%.Source: §5.7.6 · pp. 17–18
• Adversarial-abstract complaints were not much higher than complaints about benign LLM-paraphrased abstracts, while human review and iteration may reduce complaints further.Coherence and consistency problems may have benign explanations, giving colluding authors plausible deniability; the evaluated attack was fully automatic.Source: §5.7.6 · p. 18

6 Discussion

• The study finds that colluders can manipulate text-based reviewer matching with high success, while no human-study participant detected malicious intent in the modified abstracts.Participants did complain about coherence or consistency, but benign LLM-edited abstracts elicited similar complaints, supporting plausible deniability.Source: §6 · p. 18
• Platform and algorithm changes based on the findings and recommendations have been implemented and used at a major AI/ML conference.Source: §6 · p. 18

6.1 Recommendations for mitigation

• Requiring more extensive reviewer publication archives can make it harder for colluding reviewers to align their profiles strategically with targeted abstracts.Source: §6.1 · p. 18
• Average pooling can reduce the influence of a single manipulated abstract, while 75th-percentile pooling is proposed as a compromise if max pooling yields more accurate honest matches.Source: §6.1 · p. 19
• Other recommendations include reviewer awareness, randomized assignments, and making similarity scores more robust to adversarial abstract manipulation.The authors note a potential robustness–match-quality trade-off; prior evaluations and deployments of randomized assignments found little cost to match optimality.Source: §6.1 · p. 19

6.2 Limitations

• The analysis covers one colluding reviewer among 3–6 reviewers, omits common assignment constraints, and does not fully replicate real colluders or reviewer behavior.The authors also call for evaluating attack success at other venues.Source: §6.2 · p. 19

• The research identifies ML/AI conference venues, submission authors, and the research community as its key stakeholders.Source: Ethical considerations · p. 20
• The authors informed the relevant conference-management platform, which implemented safeguards that are now in use.They also argue that intent is difficult to prove from abstract content or profile changes, making proactive mitigation important.Source: Ethical considerations · p. 20
• Aggregate attack success does not establish that high-ranking historical reviewer–paper pairs involved collusion or a reviewing relationship.Source: Ethical considerations · p. 20
• The study received independent IRB approval, used deception with subsequent debriefing, and shares only anonymized, aggregated participant statistics.Source: Ethical considerations · p. 20
• Released datasets were already public, and modified abstracts are labeled; the paper says these choices keep added risks low.The authors also cite the abstracts’ availability on OpenReview or arXiv and their CC0 licenses.Source: Ethical considerations · p. 20

Open Science

• The research artifacts are available on Zenodo.Source: Open Science · p. 20
• Released adversarial examples include human-in-the-loop and fully automatic abstracts across several evaluation settings.The release also includes the 914 fully automatic abstracts used in the human-subject experiment.Source: Open Science · p. 21
• The authors release the NeurIPS 2022 and 2023 datasets, the arXiv dataset and control abstracts, and code and evaluation scripts.Source: Open Science · p. 21

LLM Prompts

• The release includes six LLM prompts and four few-shot examples, but participant-review data is limited to anonymized, aggregated statistics.The stated reason is that personalized paper assignments could compromise participant privacy even in anonymized raw reviews.Source: LLM Prompts · p. 21

A Adversarial Abstract Modification Algorithms

• The helper checks test abstract coherence, compare similarity against the current version, and optionally stop when the reviewer is already most similar in a proxy reviewer set.The early-stopping heuristic can use the previous conference’s reviewer pool; it trades off attack effectiveness against modification strength.Source: A Adversarial Abstract Modification… · pp. 23–24

A.1 The IncludeThemes Algorithm

• IncludeThemes adds themes from the reviewer’s archive to the original abstract, either through incremental human edits or an LLM-generated version.Its optional early-stopping heuristic can halt modifications when the colluding reviewer is already the most-similar reviewer in the previous conference’s pool.Source: §A.1 · pp. 24–25, A Adversarial Abstract Modification… · p. 24

A.2 The InsertKeywords Algorithm

• InsertKeywords adds keywords from the reviewer archive to an IncludeThemes abstract, using either manually reviewed insertions or an LLM-generated draft.The operation repeats for M rounds and returns the version with the highest similarity among the generated versions.Source: §A.2 · p. 26

A.2.1 FindKeywords Subroutine

• FindKeywords greedily tests candidate words and keeps words that increase SPECTER similarity, returning up to K keywords; more sophisticated attacks are possible.The search narrows candidates using the similarity increase from appending each word, and the authors report that these simple heuristics already break current systems successfully.Source: §A.2.1 · pp. 26–27

B Attack Budgets

• Success rates generally rise with N IncludeThemes rounds, with diminishing returns as stochastic paraphrases provide more attempts to select from.This experiment uses M = 0 and K = 0; Figure 6 reports the varying-N success rates and their standard-error band.Source: B Attack Budgets · p. 28, Figure 6 · p. 28
• At fixed total keyword count, smaller batches across more InsertKeywords iterations outperform larger batches across fewer iterations.Success rates increase with M × K; gradual updates let the greedy search adapt to changes in the abstract from earlier batches.Source: B Attack Budgets · p. 28, Figure 7 · p. 28

C Similarity Scores and Rankings

• For each paper, the authors rank non-author reviewers by similarity and average scores at each rank across papers in the NeurIPS 2023 dataset.This comparison connects relative rankings with absolute similarity scores.Source: C Similarity Scores and Rankings · p. 29
• Mean similarity scores decline steeply at the highest and lowest ranks, so only a small fraction of reviewers have exceptionally high or low similarity to a paper.Figure 8b zooms in on selected ranks from 1 to 1001.Source: C Similarity Scores and Rankings · p. 29

D Similarity Scores and Attack Results

• Attack success rates are generally lower for pairs in the bottom natural-similarity quartile than the top quartile, though attacks can still succeed considerably; results are mixed at rank 20.Table 7 compares top- and bottom-quartile groups with the baseline rates from Section 5.3; the authors suggest rank-20 attacks are highly successful for both groups.Source: D Similarity Scores and Attack Results · p. 29, Table 7 · p. 30

E.1.1 Human-in-the-loop Mode

• In human-in-the-loop IncludeThemes, an LLM adds a short introduction relating the abstract to themes in the reviewer’s prior work, which authors manually edit for coherence and consistency.The prompt asks for an addition under 40 words and excludes titles of prior works.Source: §E.1.1 · p. 30

E.1.2 Automatic Mode

• The automatic IncludeThemes prompt requests one sentence about inspiration from prior-work themes and a matter-of-fact style, addressing concerns about incoherent or scientifically false generations.The prompt also asks to remove personally identifiable information from the abstract.Source: §E.1.2 · pp. 30–31

E.2 The InsertKeywords Prompt

• The automatic InsertKeywords prompt asks the LLM to place keywords according to general-English or abstract-related meanings, while allowing out-of-scope terms to be rejected with explanations.It also requests a professional, rigorous tone and says not to alter content specific to the paper.Source: §E.2 · p. 32

E.3 Prompts used in Section 5.7 (Human Subject Experiment)

• The prompts were tuned separately on NeurIPS 2023 test data and follow the same rules as the earlier automatic-mode prompts.The IncludeThemes and InsertKeywords rules match those in Appendices E.1.2 and E.2, respectively.Source: §E.3 · p. 32
• IncludeThemes asks for one restrained sentence explaining inspiration from supplied related works, without naming titles, and replaces identifiable information with a de-identification placeholder.The edited title should remain unchanged, and the answer should follow the specified JSON schema.Source: §E.3 · p. 32
• InsertKeywords places relevant keywords in the abstract based on general-English or technical meaning, avoiding its first and last sentences.The prompt also asks for additional instances of keywords already present in the abstract.Source: §E.3 · p. 33
• The keyword prompt excludes unrelated terms with a 20-word explanation but requires inclusion of broadly general keywords.Non-excluded keywords must appear in the edited abstract, while the title remains unchanged.Source: §E.3 · p. 33