Source-linked AI summary
Sentiment of Emojis
Petra Kralj Novak, Jasmina Smailović, Borut Sluban, Igor Mozetič
TL;DR
The paper addresses the lack of systematic emotional-content information for the increasingly used emoji vocabulary. It constructs a context-based sentiment lexicon from sentiment-annotated multilingual tweets and finds that emojis are predominantly positive, with emoji-containing tweets differing significantly in sentiment from tweets without emojis. The resulting Emoji Sentiment Ranking is proposed as a European language-independent resource for automated sentiment analysis.
Problem
Emoji sentiment information had not been systematically analyzed or made available as a resource, despite emojis’ widespread use and importance in short informal texts.
Method
The authors construct the Emoji Sentiment Ranking for 751 emojis from sentiment labels on over 1.6 million tweets in 13 European languages, deriving each emoji’s sentiment from its tweet contexts.
Results
Most emojis are positive, with a mean sentiment score of +0.3, while tweets with emojis are more positive than tweets without emojis (+0.365 versus +0.106).
Takeaways & Limitations
The Emoji Sentiment Ranking provides a publicly available, European language-independent resource for automated sentiment analysis.
Takeaways & Limitations
Emoji sentiment is represented on a one-dimensional negativity-to-positivity scale, while subtler emotional aspects and emoji–text interplay remain to be explored.
Abstract
from arXiv · showhide
There is a new generation of emoticons, called emojis, that is increasingly being used in mobile communications and social media. In the past two years, over ten billion emojis were used on Twitter. Emojis are Unicode graphic symbols, used as a shorthand to express concepts and ideas. In contrast to the small number of well-known emoticons that carry clear emotional contents, there are hundreds of emojis. But what are their emotional contents? We provide the first emoji sentiment lexicon, called the Emoji Sentiment Ranking, and draw a sentiment map of the 751 most frequently used emojis. The sentiment of the emojis is computed from the sentiment of the tweets in which they occur. We engaged 83 human annotators to label over 1.6 million tweets in 13 European languages by the sentiment polarity (negative, neutral, or positive). About 4% of the annotated tweets contain emojis. The sentiment analysis of the emojis allows us to draw several interesting conclusions. It turns out that most of the emojis are positive, especially the most popular ones. The sentiment distribution of the tweets with and without emojis is significantly different. The inter-annotator agreement on the tweets with emojis is higher. Emojis tend to occur at the end of the tweets, and their sentiment polarity increases with the distance. We observe no significant differences in the emoji rankings between the 13 languages and the Emoji Sentiment Ranking. Consequently, we propose our Emoji Sentiment Ranking as a European language-independent resource for automated sentiment analysis. Finally, the paper provides a formalization of sentiment and a novel visualization in the form of a sentiment bar.
1 Introduction
Emojis extend emoticons into graphic symbols for concepts, ideas, and emotions, yet their emotional content had not been systematically mapped. This paper addresses that gap by constructing a context-based emoji sentiment lexicon and analyzing emoji use across languages and tweet contexts.
- Emojis are graphic symbols that represent facial expressions, concepts, ideas, activities, objects, and emotions in digital communication.
- No large-scale analysis of emojis’ emotional content or resource with emoji sentiment information had been provided.
- The Emoji Sentiment Ranking is the first emoji sentiment lexicon, covering 751 emojis and computing sentiment from tweets in which they occur.
- The study uses more than 1.6 million sentiment-annotated tweets in 13 European languages, with emojis appearing in 4% of the corpus.
- Prior work established emoticons as useful sentiment signals, while related studies examined graphical emoticons and emotional content in social media.
- The paper compares emoji sentiment across tweet contexts, emoji frequencies, positions, and languages, and provides a sentiment map, formalization, and sentiment-bar visualization.
2 Results and Discussion
The Emoji Sentiment Ranking assigns sentiment to 751 frequent emojis from multilingual, human-labeled tweets and visualizes their distribution and use. The results show predominantly positive emojis, greater positivity among frequent emojis, positional sentiment patterns, and no significant cross-language ranking differences.
- Emoji sentiment lexicon: The Emoji Sentiment Ranking assigns sentiment scores to 751 frequent emojis using over 1.6 million tweets in 13 European languages labeled by 83 annotators.Sentiment is computed from the tweets in which each emoji occurs, and the resulting lexicon is publicly available.
- Emoji sentiment lexicon: The lexicon was selected after comparing emoji occurrence rankings with Emojitracker, whose correlations were high and significant at the 1% level.The selected list included emojis with at least 5 occurrences because scores below that threshold were considered unreliable.
- Emoji sentiment map: The 751 emojis are prevailingly positive, with a mean sentiment score of +0.3, and the sentiment map separates negative, neutral, and positive emojis spatially.Bubble sizes represent emoji occurrences; neutral emojis can be genuinely neutral or bipolar, depending on their negativity and positivity components.
- Tweets with and without emojis: Tweets containing emojis have a significantly higher sentiment mean than tweets without emojis: +0.365 versus +0.106.Welch’s t-test rejected equal means with t = 87 and p-value ≈0.
- Sentiment distribution: More-frequent emojis are significantly more positive than less-frequent emojis.The comparison rejected the null hypothesis with t = 100 and p-value ≈0.
- Sentiment and emoji position: Emojis occur on average at two-thirds of a tweet, while negativity and positivity increase and neutrality decreases with distance from the beginning.More emotionally loaded emojis therefore tend to occur toward the end of tweets.
- Emojis in different languages: There is no evidence of significant differences between emoji sentiment rankings across the 13 analyzed European languages.The results support considering the Emoji Sentiment Ranking a European language-independent resource.
3 Conclusions
The paper presents the Emoji Sentiment Ranking as the first publicly available emoji sentiment lexicon and identifies applications and extensions for sentiment analysis. It also highlights limits of one-dimensional sentiment and the need to study emoji meaning in textual context and over time.
- The Emoji Sentiment Ranking is the first publicly available emoji sentiment lexicon.
- The 1.6 million annotated tweets support sentiment-classification models and other applications across languages.The authors report real-time election mood monitoring with Slovenian and Bulgarian models.
- Because human sentiment annotation is expensive, the lexicon could assist annotation or automatically label tweets containing emojis.
- The one-dimensional negativity-to-positivity scale omits subtler emotions and shallow semantic aspects of emojis.
- Future work should examine how emoji position and textual context jointly amplify or modify meaning, as well as how emoji sentiment evolves over time.
4 Methods
The methods define the emoji set, sentiment representation, uncertainty visualization, statistical testing, and correlation analyses used to construct and evaluate the Emoji Sentiment Ranking. The study relies on multilingual tweets, human sentiment annotations, probability estimates, and reliability and correlation measures.
- Data collection: Tweets were collected in 13 European languages through platform APIs, using geolocation-based search for selected languages and a 1% public-tweet stream sample for English.The collection period was April 2013 to February 2015.
- Emoji Unicode symbols: The emoji set follows Unicode 8 and includes single-character [So] symbols appearing in the tweets, excluding some double-character symbols tracked by other sources.The paper compares its symbol specification with Emojitracker and the Unicode Emoji Charts.
- Sentiment formalization: Sentiment is represented by three classes—negative, neutral, and positive—and an emoji’s distribution is estimated from sentiment-labeled tweets containing it.The paper defines p−, p0, and p+ as negativity, neutrality, and positivity, respectively.
- Sentiment formalization: For small samples, Laplace estimation with k = 3 uses a uniform prior before calculating the mean sentiment score ¯s = p+ − p−.The score ranges from −1 to +1, and the standard error of the mean is used to quantify uncertainty.
- Sentiment bar: The sentiment bar visualizes p−, p0, p+, ¯s, and ¯s ± 1.96sem, with colored proportions representing negativity, neutrality, and positivity.Its grey interval is centered at ¯s and bounded by the score range.
- Statistical analyses: Welch’s t-test evaluates equality of two means, while Pearson correlation compares ranked emoji properties and Krippendorff’s alpha measures annotator reliability.Welch’s test accommodates unequal variances and sample sizes; the correlation analyses use occurrence and sentiment rankings.