PaperlayerView exact revision on arXiv

Source-linked AI summary

Retractions Revisited: A Replication, Extension, and Mechanistic Evaluation of Linguistic Obfuscation Effects in Fraudulent Science

David M. Markowitz

arXiv:2610.13678v1cs.CL

TL;DR

How fraudulent science is communicated in research papers remains insufficiently examined, despite the consequences of fraud for the scientific record. This replication and extension compared fraudulent and genuine papers, tested proposed mechanisms, and evaluated language-based classification. Fraudulent papers were more obfuscated, and reality monitoring best accounted for the association.

  • Problem

    How scientific fraud is communicated in research papers remains less understood, despite fraud’s financial, reputational, and credibility costs.

    Source: Obfuscation Effects in Fraudulent… · p. 3

  • Method

    The study replicated and extended prior work by examining fraudulent and genuine paper language, testing mechanisms, and evaluating language-based classification.

    Source: Discussion · pp. 13–15

  • Results

    Fraudulent papers were more obfuscated, with reality monitoring best accounting for the association; detection was above chance (60.7%) using theory-driven features, while the full LIWC dictionary performed better (71.6%).

    Source: Discussion · pp. 13–15

  • Takeaways & Limitations

    The findings support the reliability of the linguistic-obfuscation effect and suggest fewer adjectives best account for its link with fraud.

    Source: Discussion · pp. 13–14

    The findings are not direct cause-and-effect evidence, effect sizes are small, and the classification models are likely ineffective for prospective deception detection.

    Source: Limitations and Future Directions · pp. 15–16

Abstract

from arXiv · show

Prior work suggests fraudulent scientists write their articles with higher rates of linguistic obfuscation than genuine scientists, with a possible mechanism suggesting obfuscation makes research more costly for a reader to analyze. This result has not undergone additional empirical scrutiny in the open science era, nor have other psychological mechanisms been evaluated. The current replication and extension addressed this opportunity by comparing 800 fraudulent and 800 genuine control papers (2013-2025). The results supported the linguistic obfuscation hypothesis, suggesting fraudulent papers were more obfuscated than genuine ones. A parallel mediation model evaluated three competing mechanisms, finding that reality monitoring, as measured through decreased rates of adjectives, best explained the link between fraudulent science and linguistic obfuscation. Cross-validated detection accuracy using obfuscation features (60.7%) and an elastic net of over 100 language dimensions (71.6%) were significantly greater than chance. Implications for scientific fraud and psychology of language research are discussed.

Retractions Revisited: A Replication, Extension, and Mechanistic Evaluation of Linguistic

Scientific fraud remains a concern because it carries financial, reputational, and credibility costs, while how fraud is communicated in research papers remains less understood. This study replicates and extends prior work to test whether obfuscation persists, what mechanisms may underlie it, and how well language can detect fraud.

Obfuscation Effects in Fraudulent Science

  • Fraud can carry financial, reputational, and credibility costs and distort the scientific record, even when its base rate is relatively low.Fraud is defined as manipulation, fabrication, or falsification of data.Source: Obfuscation Effects in Fraudulent… · p. 3
  • Understanding how scientific fraud is communicated in research papers remains a comparatively less-understood question in the literature.The paper frames this question alongside broader efforts to identify or mitigate academic deception.Source: Obfuscation Effects in Fraudulent… · p. 3
  • Prior work found data-fraud papers more obfuscated than papers without data-manipulation concerns and proposed that obfuscation makes false claims harder to appraise.That finding had not received additional empirical scrutiny, particularly after the replication crisis.Source: Obfuscation Effects in Fraudulent… · p. 4
  • The study asks whether obfuscation remains a fraud signal after the replication crisis, what mechanisms underlie the effect, and how well language can detect fraud.It uses four times as many cases as the original work and examines two additional mechanisms.Source: Obfuscation Effects in Fraudulent… · p. 4

Examining Scientific Fraud: Support from the Linguistic Obfuscation Hypothesis

The linguistic obfuscation hypothesis predicts that fraudulent writing differs from genuine writing in ways that make information harder to understand or scrutinize. Prior findings and related work also connect obfuscation with how readers process and respond to information.

Examining Scientific Fraud: Support from the Linguistic Obfuscation Hypothesis

  • The hypothesis links fraud with less readable writing and more abstraction, including lower rates of articles, prepositions, and quantifiers.These patterns are proposed to make deceptive speech more vague and difficult to scrutinize.Source: Examining Scientific Fraud: Support… · p. 5
  • Other proposed language signals include lower positive-emotion terms, more jargon, and more causal terms in fraudulent reports.The cited accounts connect these patterns to difficulty matching genre conventions, appearing credible, or elaborating claims without evidentiary grounding.Source: Examining Scientific Fraud: Support… · pp. 5–6
  • Markowitz and Hancock (2016) found fraudulent science had more abstraction, causal terms, and jargon, but lower readability and positive-emotion terms.Related work reports that less readable and more complex disclosures can alter information processing and responses in corporate settings.Source: Examining Scientific Fraud: Support… · p. 6
  • Obfuscated corporate values statements were associated with lower perceived warmth and morality, and exposure could increase readers’ deceptive behavior.Together, these findings characterize linguistic obfuscation as a marker of deception that can alter how readers evaluate and respond to a deceiver.Source: Examining Scientific Fraud: Support… · p. 6

The Current Paper

The paper replicates and extends prior work on how fraudulent papers are written relative to genuine papers. It adds tests of psychological mechanisms and evaluates language-based classification approaches.

The Current Paper

  • The paper revisits prior findings because their stability over time, especially after the replication crisis, was unclear.The authors frame establishing credibility of existing evidence as important to the social sciences.Source: The Current Paper · p. 6
  • The extension tests psychological mechanisms beyond the previously proposed role of references, including psychological distancing and reality monitoring.Academic references are also assessed as a possible mechanism linked to making science more difficult to appraise.Source: The Current Paper · p. 7
  • The paper evaluates fraud classification using both theory-driven obfuscation dimensions and an exploratory set of over 100 verbal features.The approaches are used to assess how well language dimensions alone discriminate fraudulent from genuine papers.Source: The Current Paper · p. 7

Method

The data collection assembled retraction records from multiple sources and combined them with PubMed metadata. The resulting corpus was filtered and deduplicated, with only a subset of papers having notices coded for fraud.

Data Collection

  • Retraction Watch records were supplemented with additional retractions identified through PubMed and the CrossRef API.The CrossRef query covered retractions communicated directly by publishers.Source: Data Collection · p. 7
  • Records were initially restricted to papers published from 2013 onward with PubMed identifiers, then merged with PubMed metadata.Retraction Watch data available as of September 2026 yielded an initial list of 23,647 records meeting those criteria.Source: Data Collection · p. 7
  • The final corpus contained 43,261 retracted papers, of which 16,663 had notices coded for fraud.The corpus was assembled by merging and deduplicating records across sources using DOI or PubMed identifier.Source: Data Collection · p. 8

Fraud Identification

Two language models classified retraction notices to identify fraudulent papers, which were then paired with genuine controls matched on journal and publication year.

Fraud Identification

  • GPT-4o and Claude Sonnet 5 classified 16,663 retraction notices as fraud, not fraud, or unclear.Agreement was almost perfect (κ = 0.945); a third model resolved 297 disagreements, identifying 1,540 papers retracted for fraud.Source: Fraud Identification · p. 8
  • Controls were matched on journal and publication year, and papers appearing in the retraction corpus or already assigned as controls were excluded.Eligible controls were selected at random.Source: Fraud Identification · p. 8
  • The final corpus contained 800 fraudulent papers and 800 genuine controls with obtainable full texts.Controls were selected from unretracted PubMed papers; 98% of pairs were also matched on topic.Source: Fraud Identification · p. 8

Automated Text Analysis

Fraud papers had higher linguistic obfuscation than genuine controls, while adjectives were the strongest mediator among the psychological mechanisms examined. The index combined five standardized language dimensions.

Automated Text Analysis

  • The obfuscation index combined five standardized language dimensions measured using LIWC and the Flesch Reading Ease metric.Its dimensions were jargon, abstraction, causal terms, positive tone, and readability.Source: Automated Text Analysis · pp. 8–9

Results

  • Fraud papers showed greater linguistic obfuscation than controls (Cohen’s d = 0.27), with higher jargon, abstraction, causal terms, and positive tone, but were more readable.Effect sizes were d = 0.23 for jargon, 0.28 for abstraction, 0.16 for causal terms, 0.28 for positive tone, and 0.35 for readability; results were largely consistent over time.Source: Table 1

Psychological Mechanisms

  • Adjectives were the only substantial mediator, and the direct effect of article type on obfuscation was no longer significant with mediators in the model.The adjective indirect effect was B = 0.24 (95% CI [0.19, 0.29]), accounting for 86% of the total effect; fewer adjectives were associated with more obfuscation. Distancing and references had negligible indirect effects (B = 0.01 and B = -0.009, respectively).Source: Psychological Mechanisms · pp. 10–11

Classification

Cross-validated language models distinguished fraudulent from genuine papers above chance. The broader LIWC feature model outperformed the five-feature obfuscation model, while open-vocabulary and embedding approaches provided additional comparisons.

Classification

  • 71.6% detection accuracy came from the elastic net of LIWC dimensions, versus 60.7% for the five obfuscation features; both exceeded chance.The elastic net had 75.1% sensitivity, 68.0% specificity, and AUC = .784; the five features accounted for approximately half of the full LIWC model’s above-chance accuracy.Source: Classification · pp. 11–12
  • The strongest predictors in the full model were fewer differentiation words, more power words, greater focus on the past, and fewer adjectives in fraudulent papers.Source: Classification · p. 12
  • The open-vocabulary model (75.6%) improved on the full LIWC model by about three percentage points, while the SPECTER embedding model (73.9%) did not differ from full LIWC.The open-vocabulary model was superior to other statistical classification approaches on AUC metrics.Source: Classification · p. 12

Open Science Characteristics of Fraudulent Papers

An exploratory analysis used ODDPub to assess open research practices in full-text articles. Fraud-retracted papers were less likely to contain open data, with no significant relationship for open code.

Open Science Characteristics of Fraudulent Papers

Discussion

The study replicates the finding that fraudulent papers are more linguistically obfuscated and extends the account by testing mechanisms and language-based detection. Its discussion highlights reduced detail as a likely mechanism, while treating readability and prospective detection as limited signals.

Discussion

  • A larger, more recent sample again showed greater obfuscation in fraudulent papers, with four of five dimensions replicating in the predicted direction and the pattern largely consistent over time.The discussion frames this as evidence for the reliability of the original finding.Source: Discussion · p. 13
  • Readability did not replicate as a marker of fraud, and the authors suggest formulaic, simple paper-mill writing as one speculative explanation.They caution that readability may be less stable than other dimensions and call for further empirical attention.Source: Discussion · pp. 13–14
  • Reality monitoring best accounted for the fraud–obfuscation link: fraudulent papers had fewer adjectives, and fewer adjectives were associated with greater obfuscation.The authors suggest this may reflect missing detail in descriptions of research lacking evidentiary grounding, rather than deliberate concealment as the primary explanation.Source: Discussion · pp. 13–14
  • 60.7% accuracy for five obfuscation features and 71.6% for the full LIWC model exceeded chance, but language patterns are not reliable enough for forensic or prospective detection.The theory-driven features offer a parsimonious, interpretable foundation, while strongest full-model predictors outside the obfuscation hypothesis point to unexplained verbal dimensions.Source: Discussion · pp. 14–15
  • Fraud-retracted papers were less likely to contain open data, linking written detail and research-material transparency as complementary signals of credibility.The discussion proposes that authors with fabricated or manipulated data have more to lose by sharing evidence.Source: Discussion · p. 15

Limitations and Future Directions

The authors qualify the study’s scope and its potential use in detecting fraud, and identify further analyses needed to assess how fraud and language vary across settings.

Limitations and Future Directions

  • The corpus is primarily biomedical, so how papers from other disciplines might reveal scientific fraud remains unclear.The authors identify cross-disciplinary analysis as a future consideration.Source: Limitations and Future Directions · p. 15
  • The findings are not direct cause and effect, and their effect sizes are small, though consistent with prior deception scholarship..Source: Limitations and Future Directions · p. 15
  • Above-chance classification is likely ineffective for prospective deception detection; improving it would require more cases and more sophisticated word-counting approaches.The limitation applies to the theory-driven and exploratory models.Source: Limitations and Future Directions · p. 16
  • Future work could test how different forms of fraud change the results, compare within-author patterns, and examine how LLM-written sections or other questionable practices alter scientific reporting.The authors also point to generative AI as a reason reporting patterns may change over time.Source: Limitations and Future Directions · p. 16

Obfuscation Results Over Time Across Language Dimensions

The figure note specifies its uncertainty intervals and how sample sizes are counted.

Obfuscation Results Over Time Across Language Dimensions

Loading 2610.13678v1…