Source-linked AI summary
Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection
David Ifeoluwa Adelani, Haotian Mai, Fuming Fang, Huy H. Nguyen, Junichi Yamagishi, Isao Echizen
TL;DR
Fake-review attacks traditionally require specialized expertise, but this paper examines whether publicly available neural language models can support a lower-skilled attack. It generates reviews from sentiment-bearing seeds with GPT-2 and filters them with BERT; humans judged them near randomly, while detectors remained imperfect. The authors therefore identify both an effective attack construction and a detection challenge, with limited context preservation and diversity.
Problem
Existing fake-review generation methods require expert-trained topic-specific language models or post-processing, motivating a lower-skilled approach to attacking online review systems.
Method
The method fine-tunes GPT-2 to generate reviews from whole reviews with desired sentiment, then uses a BERT classifier to retain outputs with matching sentiment.
Results
Humans selected real reviews at near-chance rates, while combined automatic detectors achieved a 22.5% equal error rate but remained far from perfect.
Takeaways & Limitations
Publicly available language models can support sentiment-preserving fake-review generation that challenges human judgment and existing automatic countermeasures.
Takeaways & Limitations
Generated reviews lack diversity and do not strictly preserve the original review’s context, which the authors identify as areas needing further work.
Abstract
from arXiv · showhide
Advanced neural language models (NLMs) are widely used in sequence generation tasks because they are able to produce fluent and meaningful sentences. They can also be used to generate fake reviews, which can then be used to attack online review systems and influence the buying decisions of online shoppers. To perform such attacks, it is necessary for experts to train a tailored LM for a specific topic. In this work, we show that a low-skilled threat model can be built just by combining publicly available LMs and show that the produced fake reviews can fool both humans and machines. In particular, we use the GPT-2 NLM to generate a large number of high-quality reviews based on a review with the desired sentiment and then using a BERT based text classifier (with accuracy of 96%) to filter out reviews with undesired sentiments. Because none of the words in the review are modified, fluent samples like the training data can be generated from the learned distribution. A subjective evaluation with 80 participants demonstrated that this simple method can produce reviews that are as fluent as those written by people. It also showed that the participants tended to distinguish fake reviews randomly. Three countermeasures, Grover, GLTR, and OpenAI GPT-2 detector, were found to be difficult to accurately detect fake review.
1 Introduction
Neural language models can generate fluent text but also enable low-skilled attackers to produce sentiment-preserving fake reviews using publicly available models. The proposed approach combines GPT-2 generation with BERT-based filtering, and evaluations found that humans and automated detectors struggled to identify the resulting reviews.
- 1 Introduction: The threat model obtains a review with the desired sentiment from a target website and uses it to generate many same-sentiment fake reviews.The attacker identifies suitable reviews and supplies them to a language model for automatic generation.
- 1 Introduction: The work addresses fake-review attacks by using neural language models that generate fluent, meaningful text for online-system manipulation.Such attacks can influence online shoppers and affect companies or competitors through positive or negative reviews.
- 1 Introduction: The method generates reviews with GPT-2 and filters out undesired sentiments using a fine-tuned BERT text classifier.GPT-2 is adapted to Amazon and Yelp review databases because its original training data may produce irrelevant topics.
- 1 Introduction: The reviews were as fluent as human-written reviews, while participants tended to identify fake reviews randomly.Automatic detectors performed better than humans but still had far-from-perfect accuracy.
2 Related Work
Prior automated fake-review attacks often required keyword-based generation, post-processing, or additional metadata, making them more demanding. This work instead uses whole reviews as seeds, avoids word modification, and filters outputs by sentiment.
- 2 Related Work: Earlier methods generated reviews from keywords or required contextual metadata, then needed post-processing to make the text fit the desired topic.Examples include replacing nouns or supplying shop name, location, and rating information.
- 2 Related Work: The proposed method seeds generation with a whole review and requires neither additional information nor additional processing before sentiment filtering.This makes the approach more straightforward than the related methods described.
- 2 Related Work: Because generated reviews are not modified after generation, their fluency remains close to that of the training samples.The method adapts a pre-trained model and can therefore be implemented by low-skill attackers.
- 2 Related Work: Unlike adversarial text examples, which target text classifiers through small perturbations, these fake reviews aim to change overall user impressions.The distinction concerns the target of deception: classifiers versus people.
3 Fake review generation
The method generates sentiment-preserving fake reviews with GPT-2 and filters them using a BERT classifier. Fine-tuning adapts GPT-2 to review data, while experiments evaluate multiple language models and sentiment classification.
- Sentiment-Preserving Fake Review Generation: The two-step procedure generates a review from an original seed with GPT-2, then retains it only when BERT assigns the same sentiment.Reviews with undesired sentiment are discarded, producing a sentiment-matched fake-review pool.
- Fine-tuning Language Model on Review Data: Fine-tuning the 117M GPT-2 model on Amazon and Yelp reviews made generated texts review-like after pre-trained GPT-2 sometimes produced non-review-like text.Training stopped when validation error no longer decreased.
- Sentiment validation: 96.2% Amazon and 96.0% Yelp test accuracy was achieved by the fine-tuned BERT classifier, exceeding character-level CNN results.The corresponding CNN accuracies were 94.49% and 94.11%, respectively.
4 Experiment
The experiment evaluated sentiment preservation, fluency, human discrimination, and automatic detection across generated fake reviews. Fine-tuning improved sentiment preservation and fluency, while humans and detectors struggled to reliably distinguish fake from real reviews.
- 4.1 Measurements and Setup: The evaluation generated 20 different fake reviews from each of 1,000 seed reviews for every language model and dataset.Four models were investigated: pre-trained GPT-2, fine-tuned GPT-2, mLSTM, and sentiment modeling.
- 4.3 Subjective Evaluation: 80 volunteers evaluated fluency using a 5-point MOS scale after reviewing real reviews and sentiment-matched generated reviews.The study included 39 native and 41 non-native English speakers and used 50 real reviews from each test dataset.
- 4.2 Sentiment-preserving fake review analysis: Fine-tuned GPT-2 preserved the original review’s sentiment better than pre-trained GPT-2 on both Amazon and Yelp.On Amazon, sentiment modeling had the highest rate; on Yelp, mLSTM had the highest rate but generalized poorly beyond food and restaurant reviews.
- 4.3 Subjective Evaluation: Fine-tuning improved fluency over pre-trained GPT-2, while fine-tuned GPT-2 achieved the highest fake-review score on Yelp at 3.30 MOS.Amazon scores were highest for sentiment modeling, whereas Yelp’s original reviews scored higher than its fake reviews overall.
- 4.3 Subjective Evaluation: 25.4%–34.6% correctness showed that participants selected the real review near the 25% four-option chance rate.The lowest overall correctness was 25.4% for Amazon and 20.8% for Yelp; the highest was 29.1% and 34.6%, respectively.
- 4.4 Automatic Fake Review Detection: 22.5% EER was the lowest automatic-detection error, yet the authors report that precisely detecting these fake reviews was not straightforward.This result was obtained by fusing all three detectors or fusing GLTR with GPT-2PD.
5 Conclusion
The proposed method generated sentiment-preserving fake reviews that were judged as fluent as human-written reviews, while humans and automated detectors struggled to identify them. The authors note limited diversity as a potential weakness and plan further improvements.
- 5 Conclusion: 3.23 and 3.30 mean opinion scores for Amazon- and Yelp-based fake reviews were comparable to real-review scores of 2.95 and 3.49.The fluency evaluation used 80 participants and a 1–5 scale.
- 5 Conclusion: The method combines GPT-2 generation with BERT sentiment filtering without post-processing or word modification.This design supports fluent samples while preserving the desired sentiment.
- 5 Conclusion: 20.8%–34.6% correctness was achieved when participants selected the most likely real review, roughly equivalent to random selection.Each set contained one real review and three fake reviews presented in random order.
- 5 Conclusion: 22.5% detection equal error rate was obtained by Grover, GLTR, and GPT-2 detector countermeasures.These automatic methods outperformed humans but remained far from perfect.
- 5 Conclusion: Because generated reviews are the most probable sequences, they lack diversity and may already be covered by existing countermeasures.The authors expect this distributional overlap could further increase detection errors.