Source-linked AI summary
NetSpam: a Network-based Spam Detection Framework for Reviews in Online Social Media
Saeedreza Shehnepoor, Mostafa Salehi, Reza Farahbakhsh, Noel Crespi
TL;DR
Spam reviews can mislead people who rely on online reviews, while prior approaches have not shown the importance of each extracted feature type. NetSpam models review datasets as heterogeneous information networks and uses feature weights in classification; on Yelp and Amazon, it outperforms existing methods, with review-behavioral features performing better than the other categories.
Problem
Spam reviews can mislead people relying on online reviews, and prior approaches have not shown the importance of each extracted feature type.
Source: §I · p. 1Method
NetSpam models review datasets as heterogeneous information networks and uses metapaths and feature weights to classify reviews.
Source: §I · pp. 1–2Results
On Yelp and Amazon, NetSpam outperforms existing methods, and review-behavioral features perform better than the other feature categories.
Source: §I · p. 1, §VI · p. 10Takeaways & Limitations
Using fewer features with higher weights can yield better accuracy with less time complexity.
Source: §I · p. 1
Abstract
from arXiv · showhide
Nowadays, a big part of people rely on available content in social media in their decisions (e.g. reviews and feedback on a topic or product). The possibility that anybody can leave a review provide a golden opportunity for spammers to write spam reviews about products and services for different interests. Identifying these spammers and the spam content is a hot topic of research and although a considerable number of studies have been done recently toward this end, but so far the methodologies put forth still barely detect spam reviews, and none of them show the importance of each extracted feature type. In this study, we propose a novel framework, named NetSpam, which utilizes spam features for modeling review datasets as heterogeneous information networks to map spam detection procedure into a classification problem in such networks. Using the importance of spam features help us to obtain better results in terms of different metrics experimented on real-world review datasets from Yelp and Amazon websites. The results show that NetSpam outperforms the existing methods and among four categories of features; including review-behavioral, user-behavioral, reviewlinguistic, user-linguistic, the first type of features performs better than the other categories.
I. INTRODUCTION
NetSpam addresses the difficulty of identifying spam reviews by modeling review datasets as heterogeneous information networks and weighting features for classification. It evaluates the framework on Yelp and Amazon, while the preliminaries define the network and classification concepts it uses.
I. INTRODUCTION
- Reviews that aim to change users’ perceptions are described as spam, and their spread can mislead users and affect businesses.The introduction notes that reviews influence product and service decisions and that spam reviews are often written in exchange for money.Source: §I · p. 1
- NetSpam models review datasets as heterogeneous information networks and weights spam features to label reviews using supervised or unsupervised approaches.The framework’s stated gap is the lack of a classifier that calculates each feature’s importance in determining spam reviews.Source: §I · pp. 1–2
- On Yelp and Amazon, review-behavioral features receive higher weights and perform better than the other feature categories in semi-supervised and unsupervised approaches.The authors also report no noticeable performance variation across supervision levels of 1%, 2.5%, and 5%, or with an unsupervised approach.Source: §I · p. 1
- NetSpam’s contributions include metapath-based classification and a feature-weighting method that can estimate feature importance without ground-truth labels.The paper contrasts this with earlier feature-importance approaches that depended on ground truth.Source: §I · p. 2
A. Definitions
- The network definitions distinguish node and link types, describe possible connections in a network schema, and use metapaths to express indirect relations between node types.The classification task predicts labels for unlabeled nodes of a designated target type.Source: A. Definitions · p. 2
B. Feature Types
NetSpam organizes review and user signals into behavioral and linguistic feature categories. The proposed solution uses a feature list to define the network structure for spam detection.
B. Feature Types
- The paper extends metapaths to paths connecting nodes through shared features and uses metadata to include review authors, businesses, ratings, dates, and spam labels.Source: B. Feature Types · p. 2
- Review-behavioral features use review metadata, including early time frame and threshold rating deviation, whereas review-linguistic features are extracted from review text.The linguistic examples are the ratio of first-person pronouns and the ratio of exclamation sentences.Source: B. Feature Types · p. 2
- User-behavioral features summarize an individual user’s review patterns, while user-linguistic features measure similarity between reviews written by different users.Examples include review burstiness, users’ average negative-rating ratio, and average or maximum content similarity.Source: B. Feature Types · p. 2
III. NETSPAM; THE PROPOSED SOLUTION
- The proposed-solution section presents Algorithm III.1 as the framework’s procedure.Source: §III · p. 2
A. Prior Knowledge
NetSpam initializes review spam probabilities, builds a network schema from selected spam features, and creates metapath links using discretized feature-based certainty levels. The chosen number of levels balances sparse connections against overly coarse probabilities.
A. Prior Knowledge
- Semi-supervised prior probabilities use known spam labels and treat unlabeled reviews as non-spam; the unsupervised prior averages feature-based spam probabilities.Source: A. Prior Knowledge · p. 3
B. Network Schema Definition
- The network schema is defined from the selected spam-feature list and specifies how network components connect.The example schema uses NR, ACS, PP1, and ETF.Source: B. Network Schema Definition · p. 3, Figure 1 · p. 3
C. Metapath Definition and Creation
- For each feature-based metapath, reviews are linked when their discretized spam-certainty levels match.The framework uses 20 levels; too few risk bipolar values, while too many yield more uniform probabilities and fewer connected reviews per level.Source: C. Metapath Definition and Creation · p. 3
- The classification procedure takes a review dataset, spam-feature list, and pre-labeled reviews, and returns feature importance and review spamicity probabilities.Source: C. Metapath Definition and Creation · p. 4
D. Classification
NetSpam’s classification has a weight-calculation step and a labeling step. Together, they estimate feature importance and each review’s probability of being spam.
D. Classification
- Classification first calculates weights to determine how important each spam feature is for spotting spam reviews.Source: D. Classification · p. 4
- The second classification step labels reviews by calculating each review’s final probability of being spam.Source: D. Classification · p. 4
- The two-step classification procedure combines feature-importance estimation with review-level spam-probability labeling.Source: D. Classification · p. 4
1) Weight Calculation:
NetSpam weights metapaths to rank spam features and uses those weights to estimate labels for unlabeled reviews. These weights also indicate which features may be valuable when feature computation is costly.
1) Weight Calculation:
- The resulting weights represent the importance of heterogeneous relation paths and help estimate labels for unlabeled reviews.The method assumes linked nodes may share labels and uses metapath weights in the labeling step.Source: 1) Weight Calculation · p. 4
- Metapath weights indicate which spam features rank reviews better and are used in the next labeling step.The weights can also help select valuable features when computing some features, such as linguistic features, is costly.Source: 1) Weight Calculation · p. 4
- The weights also help explain how spam reviews form and identify features that may be more valuable for detection.Source: 1) Weight Calculation · p. 4
- The proposed calculation uses metapath values between reviews and their labels in the pre-labeled set.The text defines the metapath value for a review pair and assigns each review a spam label indicator based on the pre-labeled data.Source: 1) Weight Calculation · p. 4
2) Labeling:
NetSpam estimates an unlabeled review’s spam probability from its links to other reviews. This relies on the assumption that connected reviews tend to have similar labels.
2) Labeling:
- The framework illustrates its labeling process on a review network.Source: 2) Labeling · p. 4
- More links to non-spam reviews increase the estimated probability that a review is non-spam, and links to spam reviews shift it toward spam.The method treats a review’s connections as evidence of shared features and similarity.Source: 2) Labeling · p. 4
IV. EXPERIMENTAL EVALUATION
The evaluation introduces Yelp and Amazon review datasets and describes three sampling variants derived from the Yelp data. The Yelp set supplies near-ground-truth spam labels, while Amazon is used in unsupervised evaluation without credible labels.
A. Datasets
- The Yelp dataset contains almost 608,598 NYC restaurant and hotel reviews with recommender-filtered near-ground-truth spam labels.It also includes review ratings and dates, plus reviewer and restaurant identifiers.Source: A. Datasets · p. 5
- From the main dataset, the review-based subset randomly samples 10% of reviews uniformly; the item-based subset samples 10% of each item's reviews uniformly; the user-based subset selects one per ten reviews.For users with fewer than ten reviews, sampling was changed so at least one review from every user was selected.Source: A. Datasets · p. 5
- Amazon reviews are evaluated in unsupervised mode because the dataset has no credible labels.Source: A. Datasets · p. 5
B. Evaluation Metrics
Evaluation ranks reviews by their final spam probabilities and calculates AP and AUC from that ranking. The figures compare approaches across datasets and supervision levels.
B. Evaluation Metrics
- AUC evaluates ranking using false-positive rate and true-positive rate, integrating the curve formed by those values.The described axes place TPR on x and FPR on y.Source: B. Evaluation Metrics · pp. 5–6
- The AP and AUC figures compare Random, SPeaglePlus, and NetSpam across datasets and 1%, 2.5%, and 5% supervision levels.Source: Figure 3 · p. 6, Figure 4 · p. 6
- AP uses the positions of ground-truth spam reviews in the sorted review list to evaluate the ranking.The method first identifies the ranked positions of reviews carrying spam labels.Source: B. Evaluation Metrics · p. 6
- Both metrics are calculated from the final ranking, which is based on review spam probabilities.Different supervision levels provide labels for training the weight calculation.Source: B. Evaluation Metrics · p. 6
C. Main Results
NetSpam outperforms comparison methods on the reported spam-detection results, while its feature-weight analysis finds behavioral features especially useful. Feature weights vary by dataset, and the paper reports a correlation between weights and accuracy.
1) Accuracy:
- NetSpam outperforms Random and SPeaglePlus; against SPeaglePlus, it leads on AP and AUC across four datasets, especially as feature count increases.Different supervision levels have no considerable effect on either method’s metric values.Source: 1) Accuracy · p. 6, C. Main Results · p. 6
2) Feature Weights Analysis:
- Feature weights are highest for the Main dataset, followed by Review-based, User-based, and Item-based datasets for most of the top four features.Fig. 5 compares weights across datasets and 1%, 2.5%, and 5% supervision.Source: Figure 5 · p. 7, 2) Feature Weights Analysis · p. 7
- In the Main dataset, all four highest-ranked features are behavioral; DEV ranks first and NR second across the other datasets.ETF ranks third for User-based and Review-based datasets, while PP1 ranks third for Item-based.Source: Table IV · p. 7, 2) Feature Weights Analysis · p. 7
- For the Main dataset, feature weights correlate with AP at 0.958 (p-value=0.0001).Fig. 6 plots feature weights against accuracy for the Main dataset using 5% training data; the paper also reports that AP and AUC rise as feature weights increase.Source: Figure 6 · p. 7, 2) Feature Weights Analysis · p. 7
- Behavioral categories outweigh linguistic ones; RB leads in the first three datasets, while UB leads in Main.Fig. 7 compares RB, UB, RL, and UL at 5% supervision; behavioral features perform better than other categories across supervision settings.Source: Figure 7 · p. 8, 2) Feature Weights Analysis · p. 8
- Except for the top two features, feature weights vary substantially with the dataset used to estimate them.The two most important features are the same across datasets.Source: 2) Feature Weights Analysis · p. 8
3) Unsupervised Method:
NetSpam estimates feature importance without a training set and reports correlations between feature weights and accuracy. The unsupervised analysis also finds a common ordering of the most-weighted features.
3) Unsupervised Method:
- Without a training set, NetSpam can calculate feature importance and identify a feature set associated with better performance.For the Main dataset, weight-accuracy correlation is 0.78 (p-value=0.0208) for NetSpam and 0.90 (p=0.0021) for SPeaglePlus.Source: 3) Unsupervised Method · p. 8
- Across the reported Amazon and Yelp settings, the unsupervised feature ranking is DEV, NR, ETF, then BST.The paper describes this ordering as evidence of shared behaviors among spammers and spam reviews across social networks.Source: 3) Unsupervised Method · p. 8
- For the Main dataset, unsupervised feature weights correlate with accuracy at 0.78 (p-value=0.0208) for NetSpam and 0.90 (p=0.0021) for SPeaglePlus.Fig. 8 shows the unsupervised feature-weight and accuracy relationship for the Main dataset.Source: Figure 8 · p. 8, 3) Unsupervised Method · p. 8
4) Time Complexity:
The paper gives separate offline and online complexity bounds and uses weighted metapaths to address camouflage. Its related-work discussion covers linguistic methods and frames spam detection as not fully solved.
4) Time Complexity:
- Offline processing has complexity O(e^2m), while online review classification has complexity O(em).Here e is the number of network edges or reviews, and m is the number of features.Source: 4) Time Complexity · p. 8
5) The Impact of Camouflage Strategy:
- To reduce camouflage effects, NetSpam uses weighted metapaths that retain even small feature values instead of treating metapaths as binary.The paper favors this fuzzy formulation over bipolar 0/1 values.Source: 5) The Impact of Camouflage Strategy · pp. 8–9
V. RELATED WORKS
- The related-work discussion describes spam detection as challenging and not fully solved, then organizes prior studies into three categories.One category uses linguistic patterns, including unigram and bigram features and probabilistic language modeling.Source: §V · p. 9, A. Linguistic-based Methods · p. 9
A. Linguistic-based Methods
- Linguistic-based methods use text features such as unigrams, bigrams, pairwise review features, and capitalization patterns to detect spam.The passage also reports a prior estimate that 2% of reviews on business websites were spam.Source: A. Linguistic-based Methods · p. 9
B. Behavior-based Methods
The related-work discussion contrasts behavioral and graph-based spam-detection methods with NetSpam’s feature-weight approach. The conclusion reports results on Yelp and Amazon and points to further applications.
B. Behavior-based Methods
- Behavior-based studies use reviewer metadata and patterns such as rating distributions, rate deviation, timing, and location to identify suspicious behavior.The cited approaches include supervised, trust-aware, and network-classification methods.Source: B. Behavior-based Methods · p. 9
C. Graph-based Methods
- Graph-based methods connect users, reviews, and items to rank or label reviews, while the paper identifies feature importance in classification as an unmet need.Prior examples use LBP, shared-IP assumptions, trust assumptions, and hybrid methods; the discussion says behavioral features generally outperform linguistic ones in accuracy.Source: C. Graph-based Methods · p. 9
VI. CONCLUSION
- On labeled Yelp and Amazon datasets, NetSpam’s metapath-derived weights support spam detection, including feature prioritization without a training set.The conclusion reports better performance than previous works with a small number of features, and better results for review-behavioral features than other categories in AP, AUC, and calculated weights.Source: §VI · pp. 9–10
- Future work includes applying metapaths to spammer-community detection, incorporating product features, and studying spam detection in multilayer networks.The paper characterizes information diffusion and content sharing in multilayer networks as a young research area.Source: §VI · p. 10