PaperlayerView exact revision on arXiv

Source-linked AI summary

An Improved Text Sentiment Classification Model Using TF-IDF and Next Word Negation

Bijoyan Das, Sarit Chakraborty

arXiv:1806.06407v1cs.CLcs.IR

TL;DR

TF-IDF can misclassify sentences containing negation, motivating a sentiment-classification method that combines TF-IDF with Next Word Negation. The authors report increased accuracy across three datasets and identify Linear SVM as their best-performing classifier among those tested.

  • Problem

    TF-IDF gives importance to uncommon words but can perform inaccurately on sentences containing negation, a construction that affects polarity.

    Source: §III · p. 3

  • Method

    The method combines TF-IDF word scores with Next Word Negation, which changes the very next word following a negation word.

    Source: §III · p. 3

  • Results

    TF-IDF with Next Word Negation achieved 89.91% accuracy on IMDB reviews, 88.86% on Amazon product reviews, and 96.83% on SMS spam datasets, with a stated overall accuracy increase.

    Source: §V · p. 5

  • Takeaways & Limitations

    At 10,000 features, Linear Support Vector Machine produced the highest accuracy among the three algorithms tested.

    Source: §4.3 · p. 5

    The authors identify contextual opposites of the word following a negation as a future direction for further improving accuracy.

    Source: §V · p. 6

Abstract

from arXiv · show

With the rapid growth of Text sentiment analysis, the demand for automatic classification of electronic documents has increased by leaps and bound. The paradigm of text classification or text mining has been the subject of many research works in recent time. In this paper we propose a technique for text sentiment classification using term frequency- inverse document frequency (TF-IDF) along with Next Word Negation (NWN). We have also compared the performances of binary bag of words model, TF-IDF model and TF-IDF with next word negation (TF-IDF-NWN) model for text classification. Our proposed model is then applied on three different text mining algorithms and we found the Linear Support vector machine (LSVM) is the most appropriate to work with our proposed model. The achieved results show significant increase in accuracy compared to earlier methods.

I. INTRODUCTION

The introduction motivates sentiment classification for interpreting growing streams of user posts and product opinions, then frames a study of three document representations and multiple classifiers.

I. INTRODUCTION

  • The introduction presents sentiment classification as useful for consumer feedback, medical records, legal documents, and spam email.Source: §I · p. 1
  • The study compares binary bag of words, TF-IDF, and TF-IDF with next word negation, evaluating them across machine-learning algorithms.The introduction presents next word negation as an accuracy-improving addition to the two common representations.Source: §I · p. 1
  • Documents are represented as word-value vectors: binary bag of words records presence as 1 or absence as 0, while TF-IDF uses calculated real-valued scores.The introduction describes TF-IDF scores as calculated using term frequency-inverse document frequency.Source: §I · p. 2
  • The experiments use pre-categorized IMDB movie reviews, Amazon product reviews, and SMS Spam Collection data.Source: §I · p. 2
  • Each representation is tested on the datasets, with feature size increased gradually to seek better results.Source: §I · p. 2

II. PRIOR WORK

Prior work applied varied classifiers and features to sentiment analysis, reporting performance that depended on the chosen representation, task, or text characteristics.

II. PRIOR WORK

  • SVM paired with unigrams produced the best results among the Naïve Bayes, Max Entropy, and SVM models examined by Pang, Lee, and Vaithyanathan.Source: §II · p. 2
  • Mullen and Collier reported 86.6% accuracy for hybrid SVM features based on Osgood’s theory, while noting reduced overall results with domain variability.Source: §II · p. 2
  • Zhang used SVM to classify review properties and concluded that reviews are good quality when they contain both subjective and objective information.Source: §II · p. 2
  • A fuzzy-search opinion-mining approach achieved only 72% efficiency and encountered a major problem with misspelled words.Source: §II · p. 2
  • Twitter sentiment analysis using N-gram, lexicon, and POS features achieved nearly 80% accuracy; the study also concluded POS diminished accuracy.Source: §II · p. 2
  • Farooq et al. examined syntactic and diminishing negation handling and achieved an average accuracy rate of 83.3%.Source: §II · p. 2

III. PROPOSED WORK

The proposed work represents documents with binary bag-of-words or TF-IDF features and adds Next Word Negation (NWN) to TF-IDF. Its pipeline preprocesses documents, applies NWN, forms TF-IDF features, and fits classifiers for sentiment prediction.

III. PROPOSED WORK

  • The pipeline imports domain-specific data, removes punctuation, applies NWN to punctuation-less documents, and then forms TF-IDF features.The displayed NWN algorithm omits punctuation and stop-word preprocessing for simplicity.Source: §III · pp. 2, 4
  • The TF-IDF model scores words by multiplying term frequency and inverse document frequency, unlike binary vectors that record only word presence.The paper notes that binary bag-of-words does not account for the differing importance of words such as “indulging.”Source: §III · p. 3
  • The authors state that TF-IDF emphasizes uncommon words but does not handle sentences containing negation accurately.They identify negation as a common linguistic construction that affects word or sentence polarity.Source: §III · p. 3
  • NWN marks only the word immediately following a negation, rather than every subsequent word through punctuation.For “The bird is not flying in the sky,” the transformed phrase is “not_flying in the sky.”Source: §III · p. 3
  • After feature preparation, the authors fit Linear SVM, Multinomial Naïve Bayes, and Max Entropy Random Forest classifiers to predict sentiment.The passage says the models are prepared for training and testing using the text dataset.Source: §III · p. 4

IV. EXPERIMENTAL RESULTS

The experiments compare text representations, feature sizes, datasets, and classifiers. TF-IDF with next word negation outperforms the two other representations on the movie-review dataset.

Experimental setup and Exp. 5 (Performance in different Datasets)

  • The experiments used three datasets, selected the movie-review dataset as primary, and split it into training and testing portions at a 0.8 ratio (80% training, 20% testing).The model was also tested on two datasets from different domains using 8,000 features; their accuracies are displayed in Table 6.Source: §IV · p. 4, §4.5 · p. 5

Exp. 1 (SVM with feature size from 2000-8000)

  • For the movie-review dataset, classification accuracy increases as feature size grows, with a steep rise between 2,000 and 3,000 features followed by a gradually slower increase.The experiment uses 40,000 reviews for training and 10,000 for testing.Source: §4.1 · p. 4

Exp. 2 (10-Fold Cross Validation with SVM)

  • The 10-fold cross-validation experiment evaluates each fold using 90% of the data for training and 10% for testing, with 5,000 features.The passage describes averaging fold accuracies; it gives no accuracy values.Source: §4.2 · pp. 4–5, Figure 3 · p. 5

Exp. 3 (Accuracy comparison of three algorithms)

  • At 10,000 features, Linear Support Vector Machine achieves the highest accuracy among the three classifiers.The passage refers to accuracies displayed in Table 5 but supplies no values.Source: §4.3 · p. 5, Table 5 · p. 5

Exp. 4 (10-Fold Cross Validation Comparison)

  • A separate 10-fold comparison across algorithms uses 5,000 features, but the supplied passage reports no outcome.Figure 4 is identified as the comparison visual.Source: §4.4 · p. 5, Figure 4 · p. 5

Exp. 6 (Final Comparison of Different Models)

  • At 8,000 features, TF-IDF with next word negation outperforms binary bag of words and simple TF-IDF on the movie-review dataset.The comparison is reported in Figure 5.Source: §4.6 · p. 5, Figure 5 · p. 5

V. CONCLUSION

The paper reports increased accuracy with TF-IDF and next word negation across three datasets. It identifies contextual opposites of the word following negation as future work.

Conclusion

  • TF-IDF with next word negation is reported to increase accuracy over binary bag of words and TF-IDF, with accuracies of 89.91% on IMDB, 88.86% on Amazon reviews, and 96.83% on SMS spam.The passage does not provide the comparison models’ accuracy values.Source: §V · p. 5
  • The proposed TF-IDF with next word negation approach is evaluated on IMDB reviews, Amazon product reviews, and SMS spam detection.The datasets are reported with separate accuracy values.Source: §V · p. 5
  • Future work aims to improve accuracy by considering the contextual opposite of the word following a negation word.The current approach negates only the next word.Source: §V · p. 6, Table 3 · p. 6
Loading 1806.06407v1…