Source-linked AI summary
A framework for fake review detection in online consumer electronics retailers
Rodrigo Barbado, Oscar Araque, Carlos A. Iglesias
TL;DR
Fake-review detection in consumer electronics has been insufficiently studied despite the importance of credible reviews. The paper introduces F3, constructs a four-city Yelp dataset, and evaluates classifiers, reaching an 82% F-Score and finding Ada Boost statistically superior.
Problem
Consumer electronics fake-review detection had not been studied thoroughly, despite the strong influence of credible reviews on purchases and retailer reputation.
Method
The paper proposes F3, combines review-centric and user-centric features, and evaluates them using a scraped, labeled Yelp dataset from four U.S. cities.
Results
82% F-Score was achieved, with user-centric features outperforming review-centric features and Ada Boost performing statistically better than the other tested classifiers.
Takeaways & Limitations
User behavior, especially reviewing activity and combined user-feature subsets, provides stronger fake-review detection signals than review text alone.
Takeaways & Limitations
The dataset relies on Yelp’s filter as the reference for labeling reviews as fake or trustworthy.
Abstract
from arXiv · showhide
The impact of online reviews on businesses has grown significantly during last years, being crucial to determine business success in a wide array of sectors, ranging from restaurants, hotels to e-commerce. Unfortunately, some users use unethical means to improve their online reputation by writing fake reviews of their businesses or competitors. Previous research has addressed fake review detection in a number of domains, such as product or business reviews in restaurants and hotels. However, in spite of its economical interest, the domain of consumer electronics businesses has not yet been thoroughly studied. This article proposes a feature framework for detecting fake reviews that has been evaluated in the consumer electronics domain. The contributions are fourfold: (i) Construction of a dataset for classifying fake reviews in the consumer electronics domain in four different cities based on scraping techniques; (ii) definition of a feature framework for fake review detection; (iii) development of a fake review classification method based on the proposed framework and (iv) evaluation and analysis of the results for each of the cities under study. We have reached an 82% F-Score on the classification task and the Ada Boost classifier has been proven to be the best one by statistical means according to the Friedman test.
1. INTRODUCTION
Online reviews strongly influence consumer electronics purchases, but fake reviews threaten consumers, competitors, and retailers. This study addresses the unexamined problem by proposing and evaluating a machine-learning framework across four U.S. cities.
- Motivation: Consumer electronics are heavily influenced by online reviews because products are costly, frequently updated, and extensively researched.More than 50% of consumer electronics buyers consult several word-of-mouth sources before purchasing.
- Motivation: External word-of-mouth has significant effects on consumer electronics retailers’ reputation and sales, while internal retailer reviews have limited influence.
- Motivation: Fake reviews can harm businesses, consumers, and competitors through deceptive positive or negative evaluations.
- Research gap: Prior fake-review research focused mainly on restaurants and hotels, leaving consumer electronics businesses insufficiently studied.
- Contributions: The article proposes the Fake Feature Framework, builds a four-city Yelp dataset, develops a classifier, and evaluates geographic differences in detection difficulty.
2. RELATED WORK
Prior fake-review detection research has used textual, behavioral, social, temporal, graph, and semi-supervised approaches. It has concentrated on restaurant and hotel reviews, motivating study of consumer electronics.
- Feature approaches: Fake-review detection is a deception-detection application using verbal and nonverbal clues, including textual, behavioral, social, and temporal features.
- Feature approaches: Text-only detection is challenging, while prior studies found non-textual behavioral features more relevant in restaurant and hotel domains.
- Alternative methods: Graph-based methods model relationships among reviewers, reviews, and businesses to identify suspicious reviewers.
- Alternative methods: Temporal approaches examine review bursts, spam attacks, review density, and related semantic or emotional features.
- Classifiers: Support Vector Machine was the most commonly used classifier, followed by Naive Bayes, Decision Tree, Random Forest, and Logistic Regression.
- Alternative methods: Semi-supervised and PU-learning methods address the difficulty of collecting labeled data for fake-review experiments.
- Research gap: Previous business-review studies centered on restaurants and hotels, whereas consumer electronics fake-review detection was identified as novel and relevant.
3. METHODOLOGY
The methodology scrapes Yelp reviews and user profiles from four U.S. cities, labels reviews using Yelp’s filter, and extracts review- and user-centered features through F3.
- Methodology: The pipeline scrapes Yelp data, defines and computes features, then trains a classifier to detect fake reviews.
- Scraping process: Yelp’s filter serves as the reference label because no consumer-electronics dataset was available and the filter was claimed to be highly accurate.
- Scraping process: The corpus covers reviews from New York, San Francisco, Los Angeles, and Miami, with business reviews linked to user profiles.
- Scraping process: The dataset is balanced across fake and trustful reviews and includes labeled reviews plus user social and networking information.
- Fake Feature Framework (F3): F3 organizes features into review-centric text features and user-centric personal, social, reviewing-activity, and trusting features.
- Fake Feature Framework (F3): Social features capture interactions such as friends, followers, compliments, popularity, and profile-photo presence.
- Fake Feature Framework (F3): Trusting features target abnormal behavior through review similarity, rating deviations, first reviews, and uploaded photos.
- Fake Feature Framework (F3): Review-centric features include text statistics, TF-IDF, POS tags, LDA, Word2Vec, sentiment, and emotion analysis.
4. EXPERIMENTATION AND EVALUATION
Experiments evaluate fake-review classification across feature types, cities, classifiers, and statistical tests. User-centric features outperform review-centric text features, while AdaBoost and Random Forest achieve the strongest overall performance.
- 4. EXPERIMENTATION AND EVALUATION: Ten-fold cross-validation evaluates classifiers on the scraped consumer-electronics dataset, comparing user-centric and review-centric features across cities.The experiments also assess city differences and apply statistical evaluation.
- 4.1. Review centric features: F-Score below 60% for TF-IDF bigrams makes review-centric text features a weak baseline for fake-review classification.LDA, sentiment analysis, and Word2Vec did not improve performance; Word2Vec performed similarly to random guessing.
- 4.2. User centric features: User-centric features clearly outperform review-centric features, with results structured by city, feature subset, and five classification algorithms.The user-centric groups are Social, Personal, Trusting, and Reviewing Activity features.
- 4.2. User centric features: Reviewing Activity features are most relevant and Social features least relevant, yet adding Social features improves the accuracy of other feature types.Each combination of feature types consistently increases performance.
- 4.2. User centric features: F-Scores are similar for New York, Miami, and San Francisco, whereas Los Angeles performs worse for corresponding classifier-feature pairs.Using all cities together does not surpass the best individual-city results.
- 4.4. Statistical analysis: Statistical tests reject equal classifier performance, identify significant differences involving Gaussian Naive Bayes and AdaBoost, and highlight AdaBoost and Random Forest as best performers.The Friedman F statistic is 14.5 against a critical value of 3.26; the Nemenyi critical difference is 3.05.
5. CONCLUSIONS
The paper concludes that fake reviews in consumer electronics can be detected using a framework combining review- and user-centric features, with user behavior providing the strongest signals. Results were similar across four cities, and Ada Boost performed statistically best among the evaluated classifiers.
- The F3 framework combines review-centric text features with user-centric features covering personal, social, reviewing activity, and trust information.
- Reviewing-activity features are the most relevant user-centric signals, while combining feature subsets improves overall accuracy and F-Score.
- User-centric features outperform review-centric features, whose F-Scores remained below 60% despite TF-IDF and Word2vec methods.
- 82% F-Score was the maximum result obtained with the F3 features using Random Forest or Ada Boost classifiers.
- Results were quite similar across four USA cities, although Los Angeles scored somewhat lower than New York, Miami, and San Francisco; Friedman testing favored Ada Boost statistically.
- The authors present F3 as a first attempt to organize features for fake detection and propose extending it beyond e-commerce and to tasks such as toxic-user or fake-news detection.