SentiWords: Deriving a High Precision and High Coverage Lexicon for Sentiment Analysis

1 INTRODUCTION

• Manual prior-polarity lexica have high precision but low coverage, while automatically derived lexica generally have the reverse profile.Prior polarity is the sentiment a word evokes out of context.Source: §1 · p. 1
• The paper blends SentiWordNet posterior-to-prior formulas in an ensemble that uses manually built lexica to predict prior values for unseen words.SentiWordNet assigns polarity scores by word sense, so a prior polarity must be derived from those posterior scores.Source: §1 · p. 1
• The study asks whether formula performance varies across regression and classification, SentiWordNet releases, and learning frameworks.It also tests SentiWords against individual SentiWordNet metrics, handmade lexica, and a specialized posterior-polarity lexicon across datasets.Source: §1 · pp. 1–2
• Experiments support improved posterior-to-prior computation with learning and better sentiment-analysis results using SentiWords than other available lexica.The authors report experiments in both regression and classification tasks.Source: §1 · p. 2

2 RELATED WORK

• Prior-polarity lexica support affective text modification and remain a prerequisite for compositional sentiment analysis, which combines word scores across a sentence's syntactic tree.Compositional approaches can distinguish sentences with the same words but different word order; word bigrams can capture some simple compositional phenomena such as polarity reversal.Source: §2 · p. 2
• Prior polarity also matters in copywriting, where evocative names are used without context to signal a product's competitive area and evoke semantic associations.The passage gives perfume and technology product names as examples.Source: §2 · p. 2
• Manual lexica are usually smaller and more precise, whereas automatically created resources are larger and their precision depends on the annotation algorithm.SentiWordNet-based prior-polarity formulas produce about 155,000 words, trading lower precision from automatic scoring for broad coverage.Source: §2 · p. 2
• ANEW, SO-CAL, DAL, Warriner's resource, and General Inquirer provide manually annotated sentiment ratings or labels at different lexicon sizes.The passage reports approximate sizes from about 1,000 words for ANEW to 14,000 for Warriner's resource; General Inquirer covers approximately 4,000 sentiment-bearing words.Source: §2 · pp. 2–3
• Emotion resources range from labeled WordNet synsets and concepts to word-level emotion labels or scores, with DepecheMood covering about 37,000 words.DepecheMood and the Affect database are identified as the listed resources providing emotion scores rather than only labels.Source: §2 · p. 3

3 PROPOSED APPROACH

• The approach evaluates prior-polarity formulas in regression and classification, then tests whether the ensemble lexicon improves sentence sentiment analysis over individual SentiWordNet metrics and handmade lexica.The sentence datasets cover news headlines and movie reviews, representing different forms of language use.Source: §3 · p. 3

3.1 SentiWordNet

• SentiWordNet assigns positive and negative posterior-polarity scores to synsets, which share a meaning across lemma#PoS#sense-number entries.Its scores are assigned by a classifier committee trained on WordNet glosses; the positive and negative synsets are expanded through the WordNet hierarchy.Source: §3.1 · p. 3
• Different senses of a lemma#PoS can carry different or mixed polarities, motivating the derivation of one prior polarity from all its sense scores.The first five cold#a entries illustrate these combinations.Source: §3.1 · p. 3, Table 1 · p. 3

3.2 Prior Polarities Formulae

• Each formula independently aggregates positive and negative scores across a lemma#PoS's senses, then maps the two aggregates to one prior polarity.The experiments use SWN1 and SWN3, whose releases differ in WordNet version, score adjustment, and gloss annotations.Source: §3.1 · pp. 3–4, §3.2 · p. 4
• New strength-ordered variants prioritize strongly valenced senses, while other variants omit neutral senses or aggregate by median or maximum.The formulas also include first-sense, mean, and frequency-weighted approaches; w1 weights senses geometrically, and w2 uses a harmonic series.Source: §3.2 · p. 4
• The prior-polarity formulae were evaluated against two sentiment-lexicon gold standards, one for regression and one for classification, both separately and combined in a learning framework.The basic random baseline returns a value between -1 and 1; the SWN-informed random baseline uses scores from a random sense for the given lemma#PoS. Classification also includes a baseline that always outputs the most frequent class.Source: §3.2 · p. 4

3.3 Learning Algorithms

• A fusion learner combines complementary formula predictions by learning how to map their outputs to a single prior-polarity score.The tested non-parametric learners are SVMs and Gaussian Processes; SVM experiments use grid search in 10-fold cross-validation and randomized feature selection.Source: §3.3 · p. 5

4 HUMAN-ANNOTATED SENTIMENT LEXICA

• ANEW and General Inquirer serve as human-annotated gold standards for evaluating prior-polarity formulas in regression and classification, respectively.Both provide judgments for words presented without context.Source: §4 · p. 5, §4.1 · p. 5, §4.2 · p. 6

4.1 ANEW

• ANEW provides normative ratings for roughly 1,000 words, and the experiments use its valence ratings as the regression target.ANEW also reports arousal and dominance; its valence metrics are the average annotator score and the variance in annotator scores.Source: §4.1 · p. 5

4.2 General Inquirer

• The study uses General Inquirer's positive and negative categories—4,206 words total—as binary human-assigned prior-polarity labels.The resource's latest revision contains 11,789 words tagged with 182 semantic and pragmatic labels, as well as part of speech.Source: §4.2 · p. 6

5 PRIOR POLARITIES EXPERIMENTS

• After PoS alignment and filtering, experiments retained 961 ANEW and 2,557 GI entries, using five repeated 70/30 training-test splits.Regression was evaluated with MAE and Pearson correlation; classification used accuracy and Cohen’s kappa.Source: §5 · p. 6

5.1 Discussion

• All SentiWordNet approaches significantly outperform the random baseline for both MAE and accuracy (p < 0.001).The random approach is the worst-performing metric in Tables 2–5.Source: §5.1 · p. 6
• SWN3 improves over SWN1: MAEµ 0.398 vs. 0.366 and accuracyµ 0.710 vs. 0.771.The authors partly explain this by fewer lemma-PoS entries with all-zero senses in SWN3: 48 versus 137 for ANEW and 69 versus 223 for GI.Source: §5.1 · pp. 6–7
• Formulae differ significantly in both tasks (p < 0.001), and the new posterior-polarity-saliency formulae rank among the best.The authors identify room to explore further formula variants.Source: §5.1 · p. 7
• Single-sense approaches have results similar to random-sense SWN and remain significantly below the best formulae (at least p < 0.01).Taking the first sense sometimes performs no better than choosing a random sense.Source: §5.1 · p. 7
• MAEµ with SWN1 0.366 vs. 0.391 and SWN3 0.333 vs. 0.359; accuracyµ with SWN1 0.743 vs. 0.719 and SWN3 0.792 vs. 0.781.The regression and SWN1 classification gains were significant (p < 0.001); the SWN3 classification difference was not significant (p = 0.07).Source: §5.1 · p. 7
• SVMs outperform GPs in regression (p < 0.001), while feature-selected SVMs perform best across experiments.Selected features span different formula groups, and even the worst-performing formulae can add information.Source: §5.1 · p. 8
• SV Mfs using SWN3 outperforms all other methods for prior-polarity computation from SentiWordNet.The discussion identifies it as the best-performing method overall.Source: §5.1 · p. 8

6 ERROR ANALYSIS

• 76% of the inspected errors were attributed to incorrect SentiWordNet values, based on annotators’ review of 50 errors (Cohen’s kappa = .75).Across 652 misclassified lemma-PoS, 67% were mislabeled by both methods; w2 and SV Mfs mislabelled 19% and 14%, respectively.Source: §6 · p. 8
• When the first sense conflicts with the gold label, w2 usually errs because it heavily weights that sense, while SV Mfs can be more robust to noisy scores across senses.For about 28% of misclassified words, errors in a single-sense lemma or across all its senses leave both methods unable to find the correct label.Source: §6 · p. 8
• When the first sense matches the gold label but most other senses have the opposite sign, SV Mfs usually errs while w2 does not.For confident#a, the classifier is swayed by the strong negativity of the last two senses and assigns a negative label.Source: §6 · p. 9
• On average, SV Mfs is less precise on extreme ANEW values, where the training set contains fewer examples.Figure 1 plots MAE across ANEW bins, with a horizontal line marking the overall average MAE.Source: §6 · p. 9, Figure 1 · p. 9

7 SentiWords

• SentiWords uses SV Mfs trained on Warr to score SWN3 lemma-PoS entries, then merges those scores with Warr.The training set contains 13,915 Warr entries; the model scores roughly 40,000 SWN3 lemma-PoS with at least one nonzero value.Source: §7 · p. 9
• The resulting lexicon is designed to maximize both precision and coverage.The authors describe it as a lexicon produced from lessons learned in their earlier experiments on prior-polarity computation.Source: §7 · p. 9
• SentiWordsbin was also created for experiments by training on the complete GI list.Source: §7 · p. 9

8 PRIOR POLARITIES AND SENTIMENT ANALYSIS

• The experiments compare SentiWords with four human-annotated lexica to assess the role of coverage and precision in regression and classification.ANEW and GI are low-coverage gold standards; Warr has high coverage, while Stanf has high coverage but contains posterior polarities.Source: §8 · p. 9

8.1 Datasets

• Two sentence datasets cover distinct language settings: news headlines with simplified syntax and movie-review sentences with natural language use.Both datasets are used for regression and classification experiments.Source: §8.1 · pp. 9–10

8.1.1 SemEval

• SemEval contains 1,000 news headlines annotated for emotion, and its simplified syntax makes compositional effects less prominent.The headlines average 7.21 words (5.4 lemma-PoS); only one headline contains no word present in SentiWords.Source: §8.1.1 · p. 10
• On SemEval, Warr has higher coverage than Stanf despite having fewer entries, while ANEW and GI have poor coverage and leave nearly half of sentences without a recognized sentiment word.Source: §8.1.1 · p. 10, Table 7 · p. 10

8.1.2 Sentiment Treebank

• The Stanford Sentiment Treebank contains 11,855 movie-review sentences and 215,154 annotated phrases, with each phrase judged by three annotators.The sentences average 20.4 words (11 lemma-PoS) after lemmatization and PoS tagging.Source: §8.1.2 · p. 10
• On STB, ANEW and GI have poor coverage, leaving about 40% of sentences without a recognized sentiment word; Stanf matches SentiWords coverage despite being much smaller.Stanf’s coverage matches because it was built from words in STB and restricted to entries alignable with SWN.Source: §8.1.2 · p. 10, Table 8 · p. 10

9 SENTIMENT ANALYSIS EXPERIMENTS

• The sentence-level evaluation averages recognized word scores for regression and uses majority vote for classification, with experiments run both with and without stop words.The regression measure is Pearson correlation; neutral sentences are excluded from classification.Source: §9 · pp. 10–12
• ρ values for ANEW are more than double SentiWords’ in all regression experiments (p < 0.001), despite ANEW’s high precision and low coverage.The authors attribute ANEW’s poor performance to its small size and the resulting undecidable sentences.Source: §9.1 · p. 12
• SWN formulae beat ANEW in regression and GI in classification, but have lower precision than Warr and SentiWords.Source: §9.1 · p. 13
• On STB with stop-word removal, SentiWords reaches ρ = 0.402 versus Warr’s 0.384 (p = 0.05); on SemEval, their performance is comparable.The paper reports that SentiWords helps on datasets where Warr has lower coverage.Source: Table 10 · p. 11, Table 9 · p. 11, §9.1 · p. 13
• SemEvalbest performs worse than Warr and SentiWords in regression, and worse than fbest and SentiWords in classification.The authors note that SemEvalbest uses compositional strategies but a poorer lexicon than SentiWords.Source: §9.1 · p. 13
• SentiWords is reported as a new state-of-the-art prior-polarity lexicon, outperforming other SWN formulae and handmade lexica through wide coverage and its Warr foundation.Source: §9.1 · p. 13

• The ensemble leverages manually built lexica to better predict prior polarity values for unseen words.Source: §10 · p. 13
• SentiWords combines very high coverage with good precision.Source: §10 · p. 13
• SentiWords outperformed both single SentiWordNet metrics and popular manually annotated sentiment lexica in sentiment-analysis tasks.Source: §10 · p. 13

• Lorenzo Gatti is a Ph.D. student at the University of Trento, funded by Fondazione Bruno Kessler, with interests including sentiment analysis, humor generation, and persuasive language.Source: BIOGRAPHIES · p. 15
• Marco Guerini is a computational linguistics researcher at FBK-Irst whose work focuses on persuasive communication, sentiment analysis, and social media.Source: BIOGRAPHIES · p. 15
• Marco Turchi is a tenure-track researcher at FBK-Irst whose interests include sentiment analysis and machine learning for machine translation.Source: BIOGRAPHIES · p. 15