Source-linked AI summary
The Evolution of Sentiment Analysis - A Review of Research Topics, Venues, and Top Cited Papers
Mika Viking Mäntylä, Daniel Graziotin, Miikka Kuutila
TL;DR
The rapid growth of sentiment analysis makes the field difficult to track systematically. This paper analyzes 6,996 Scopus papers through computer-assisted literature review, text clustering, qualitative coding, bibliometrics, and citation analysis, finding roots in public-opinion and subjectivity research, a nearly 50-fold rise from 2005 to 2015, and expansion across topics and data sources.
Problem
The field’s large and rapidly growing literature makes it difficult for individual researchers to keep track of its activities, while conventional reviews typically cover only one subarea.
Method
The paper conducts a computer-assisted literature review of 6,996 papers using Scopus data, automated text clustering with LDA, qualitative coding, bibliometric analysis, and citation analysis.
Results
Sentiment analysis grew nearly 50-fold from 101 papers in 2005 to 5,699 in 2015, with research spanning diverse data sources, methods, and application areas.
Takeaways & Limitations
The field has shifted beyond online product reviews toward social-media texts and applications including finance, elections, health, software engineering, and cyberbullying.
Takeaways & Limitations
The study’s literature results rely primarily on Scopus, with minor additions from Google Scholar, so database choice constrains coverage.
Abstract
from arXiv · showhide
Sentiment analysis is one of the fastest growing research areas in computer science, making it challenging to keep track of all the activities in the area. We present a computer-assisted literature review, where we utilize both text mining and qualitative coding, and analyze 6,996 papers from Scopus. We find that the roots of sentiment analysis are in the studies on public opinion analysis at the beginning of 20th century and in the text subjectivity analysis performed by the computational linguistics community in 1990's. However, the outbreak of computer-based sentiment analysis only occurred with the availability of subjective texts on the Web. Consequently, 99% of the papers have been published after 2004. Sentiment analysis papers are scattered to multiple publication venues, and the combined number of papers in the top-15 venues only represent ca. 30% of the papers in total. We present the top-20 cited papers from Google Scholar and Scopus and a taxonomy of research topics. In recent years, sentiment analysis has shifted from analyzing online product reviews to social media texts from Twitter and Facebook. Many topics beyond product reviews like stock markets, elections, disasters, medicine, software engineering and cyberbullying extend the utilization of sentiment analysis
1 Introduction
Sentiment analysis grew from public-opinion studies and computational subjectivity analysis into a broad field enabled by subjective texts on the Web. This review addresses the field’s rapid expansion by mapping its volume, venues, topics, and influential papers.
- Concept and origins: Sentiment analysis detects subjective information, traditionally classifying opinions as positive, neutral, or negative toward products or services.The paper distinguishes emotionally loaded sentiments from opinions, although the terms are often used synonymously.
- Field growth: 99% of the nearly 7,000 sentiment-analysis papers were published after 2004, marking exceptionally rapid field growth.The paper links modern expansion to the availability of large quantities of online opinions.
- Review scope: The review uses a computer-assisted literature review and bibliometric study to help researchers navigate an increasingly large body of work.It combines automated text clustering with manual qualitative analysis to characterize the field.
- Review scope: The paper characterizes sentiment analysis through research questions on publication volume, venues, topics, and highly cited studies.These questions are intended to provide an overview for newcomers and useful orientation for experienced scholars.
2 Research Methods
The study combines Scopus-based literature retrieval with bibliometric analysis, text mining, LDA topic modelling, qualitative coding, and citation analysis. These methods were chosen to analyze a literature set too large for complete manual review.
- Searching literature: The review retrieves literature primarily from Scopus, supplementing Google Scholar for highly cited papers and early studies absent from Scopus.Google Scholar results were collected through 11 queries, while Scopus supplied the main literature dataset.
- Searching literature: Search strings covering sentiment analysis, opinion mining, and historical terms were matched in titles, abstracts, or keywords using OR.The search strategy incorporated terms from an influential prior review and additional historical terminology.
- Venues and citations: Publication venues were analyzed after cleaning conference-proceedings names, and top-cited papers were identified using citation counts normalized for time.The venue analysis addressed inconsistent conference naming across years.
- Topic modelling: The authors used word clouds and LDA topic modelling because nearly 7,000 papers made manual analysis of every paper impractical.LDA models documents as mixtures of multiple topics rather than assigning each paper to only one topic.
- Topic modelling: 108 topics were selected as optimal by log-likelihood and then grouped through qualitative coding and hierarchical clustering for interpretability.The qualitative analysis examined the papers assigned to topics and organized them into higher-level groups.
3.1 Number of papers and a brief history of sentiment analysis (RQ1)
Sentiment analysis developed from public-opinion research and computational subjectivity analysis into a rapidly expanding field. Publication growth accelerated after the mid-2000s, alongside the emergence of modern Web-based sentiment analysis.
- Historical roots: The first paper matching the review’s search strings appeared in 1940 and addressed public-opinion analysis.Earlier public-opinion studies relied mainly on survey-based methods and focused on public or expert opinions.
- Computational precursors: Computational subjectivity analysis emerged in the 1990s, including Wiebe’s methods for detecting subjective sentences and a later gold standard.The Association for Computational Linguistics community is identified as influential in the birth of computer-based sentiment analysis.
- Publication growth: By 2000, only 37 papers had been published, increasing to 101 by 2005 and 1,039 by 2010.The yearly counts show momentum building around 2005.
- Publication growth: 5,699 papers had been published by 2015, while the full dataset reached 6,996 papers including many 2016 publications.The review describes this as a nearly 50-fold increase from 2005 to 2015.
3.2 Publication Venues (RQ3)
Sentiment analysis papers are distributed across many venues, with conference proceedings dominating the top rankings and the largest venues accounting for only a minority of the field. Venue shares also vary substantially, and publication output differs from citation-oriented rankings.
- Venue distribution: 11 of the top 15 venues are conference proceedings, while 4 are journals.The venue classification reflects the predominance of conference-based publication in computer science.
- Venue distribution: 684 sentiment analysis papers appeared in LNCS, nearly four times the output of the second-ranked venue.LNCS hosts 12.0% of the sentiment analysis dataset, despite less than 1% of its 2015 papers focusing on sentiment analysis.
- Venue distribution: The top two venues, LNCS and CEUR-WS, account for half of the sentiment analysis papers among the top 15 venues but only about 15% of all Scopus-indexed papers.This indicates a skewed distribution toward the leading venues within the ranking, while the field remains dispersed overall.
- Venue-specific shares: Procesamiento del Lenguaje Natural had the highest 2015 venue share, with 28.6% of its papers devoted to sentiment analysis.The journal is open access and publishes natural language processing research in English and Spanish.
- Venue-specific shares: In 2015, sentiment analysis represented 11.5% of RANLP papers, 9.5% of CCIS papers, 7.3% of EMNLP papers, and 1.6% of CIKM papers.These venue-level shares are reported relative to each venue’s total 2015 publication output.
- Publication strategy: Citation-oriented venue rankings differed from publication-output rankings, highlighting EMNLP, Expert Systems with Applications, ACL, Decision Support Systems, and WWW.The paper advises authors to choose publication strategies based on sentiment-analysis alignment, potential impact, or both.
3.3 Citation patterns (RQ2)
Citation activity grew alongside the expanding sentiment-analysis literature, but citation counts are strongly shaped by publication age. The study also compares citation patterns with software engineering and documents the field’s recent emergence.
- Cumulative and annual citation counts increased alongside paper counts, with recent annual data normalized because citations accumulate over time.The authors caution that time correction may be insufficient for papers published late in 2015.
- 54% of papers had no citations, 13% had one citation, and 33% had two or more citations.The corresponding software-engineering figures were 43%, 14%, and 43%.
- After adjusting for publication age, software engineering had more uncited papers among studies five to nine years old: 43% versus 33% for sentiment analysis.
- The most-cited sentiment-analysis paper surpassed every software-engineering paper despite the field’s recent emergence.Only five of sentiment analysis’s top 30 cited papers were published before 2005, compared with all 30 software-engineering papers.
3.4 Areas of research – Word clouds (RQ4)
Word-cloud analysis identifies social, online, review, media, and product terms as central to sentiment-analysis research. It also reveals business domains, multiple languages, and methodological terms, while comparing recent and earlier publications.
- “Social,” “online,” “reviews,” “media,” and “product” are the most prominent concepts in sentiment-analysis paper titles.
- The titles also represent movie, news, political, stock, and financial domains, alongside Chinese and Arabic language research.
- Neural, fuzzy, and supervised methods appear among the prominent methodological terms in the title corpus.
- The comparison cloud contrasts papers from 2014–2016 with papers from 2013 and earlier because early-year publication counts were too limited for decade comparisons.
3.5 Areas of Research - Topic modelling and human based classification of topics (RQ3)
Topic modelling and qualitative coding organize sentiment-analysis research into a taxonomy spanning data sources, analysis tools and techniques, and research goals. The authors report uneven topic coherence and use multiple classes for some incoherent topics.
- Topic modelling: Topic modelling identified 108 topics as optimal by log-likelihood, which were then grouped through qualitative coding.
- Topic modelling: Topic quality fluctuated: some topics were coherent, while others were incoherent and were coded under multiple classes.
- Classification tree: The highest-level taxonomy contains Data, Data Analysis, and Goal groups.
- Data and Data Analysis: Data covers information sources and includes a Different Languages subclass, while Data Analysis contains Tools and Techniques subclasses.
- Techniques and Goals: Techniques divide into Machine Learning, Natural Language Processing, and additional technique groups, while Goals divide into Application Domain and Human and Behavior orientations.
- Application domains: Application-domain goals include Society, Security, and Travel among six identified classes.
3.6 Highest cited papers (RQ5)
The highest-cited literature includes reviews, early online-review studies, Twitter research, tools and lexicons, and other works. Citation rankings show both the foundational role of reviews and the field’s movement toward social-media analysis.
- The top-cited papers are classified into five groups: reviews and overviews, early online reviews, Twitter, tools and lexicons, and others.
- Literature reviews and overviews: Four top-cited papers are literature reviews or overviews, with Pang and Lee’s review ranking highest by citations per year in both databases.
- Literature reviews and overviews: Feldman’s overview covers document- and sentence-level analysis, lexicon acquisition, and aspect-based sentiment analysis.
- Literature reviews and overviews: Bing Liu’s 167-page book spans document, sentence, and aspect-based sentiment analysis and organizes research problems with available knowledge.
- Early years – online reviews: Early online-review studies occupy seven of the top 20 positions even after ranking by citations per year.
- Representative approaches: The reviewed literature includes feature-based review summaries, phrase-level polarity classification, PMI-based semantic orientation, and machine-learning sentiment classifiers.
- Twitter, tools, and lexicons: Three top-cited papers focused on Twitter, while top-cited tools and lexicons included systems for short social-web texts and automatically generated sentiment resources.
4 Limitations
The review’s scope and interpretation are constrained by its database choice, search-string inclusions, and researcher-influenced qualitative coding.
- Search scope: The search strings included papers that used sentiment analysis mainly to motivate other work, although the authors believe such papers were a minority.Among a sample of 17 top-cited reviews or original works, all were directly about sentiment analysis.
- Database coverage: Using Scopus with minor Google Scholar additions may limit the study’s coverage of sentiment-analysis literature.The authors identify database selection as a limitation and compare Scopus with Google Scholar and Web of Science.
- Qualitative coding: Qualitative coding is influenced by the researchers who create the classification.Both authors’ software-engineering backgrounds likely influenced the resulting qualitative classification.
5 Discussions and Conclusions
The study maps sentiment analysis through a computer-assisted review of 6,996 papers, tracing its growth, venues, citations, and research topics. It finds explosive expansion, dispersed publication outlets, increasing citation impact, and a shift from product reviews toward social media and diverse applications.
- Contributions: The review analyzed 6,996 papers using automated text clustering, manual qualitative analysis, and bibliometric methods.It examined the field’s history, impact and trends, publication venues, research topics, and most-cited works.
- History and growth: Sentiment analysis grew from public-opinion studies into a major research topic after online product reviews became available in the mid-2000s.The number of papers rose from 101 in 2005 to nearly 5,699 in 2015, an almost 50-fold increase.
- Impact: Citation counts increased alongside paper counts, indicating impact at least when measured by citations.The top-cited sentiment-analysis paper exceeded citation counts of papers from the larger and more mature software-engineering comparison area.
- Publication venues: The top 15 publication venues contained only approximately 30% of all sentiment-analysis papers, showing that publications were distributed across many venues.A comparison study listing only journals was narrower because just four of this review’s top 15 venues were journals.
- Research topics: Research topics spanned newspapers, tweets, photos, chats, machine learning, natural language processing, sentiment-specific methods, and many application areas.Applications included movies, travel, health, argumentation, audience interaction, elections, expertise, sarcasm, and spam.
- Research topics: Recent studies emphasized Twitter, Facebook, mobile devices, stock markets, and human emotions, whereas earlier work focused more on product reviews, product features, and elections.The study presents a comprehensive taxonomy intended to provide broad overviews of areas too large for traditional literature reviews.