Source-linked AI summary
Sentiment Analysis in the News
Alexandra Balahur, Ralf Steinberger, Mijail Kabadjov, Vanni Zavarella, Erik van der Goot, Matina Halkia, Bruno Pouliquen, Jenya Belyaeva
TL;DR
News opinion mining lacks the clearly defined targets and annotation boundaries common in reviews, especially when news context can be mistaken for sentiment toward entities. The paper defines targets, separates news value from target sentiment, and evaluates lexicons, local windows, and category-word removal on English news. The authors report good results when the task is clearly defined, with the best tested configuration reaching 82% accuracy.
Problem
News opinion mining requires distinct treatment because target identification, news content, explicit sentiment, and article perspectives are not resolved by approaches developed mainly for subjective text types.
Method
The paper mines opinions about entities in English news using sentiment lexicons and word windows while removing subject-domain words that overlap with sentiment vocabulary.
Results
The best results used JRC Tonality and MicroWN in a six-word window, achieving 82% accuracy.
Takeaways & Limitations
The authors conclude that clearly defining the task and separating good or bad news content from target sentiment produces good results.
Takeaways & Limitations
The approach frequently classifies implicitly expressed or idiomatic sentiment as neutral because no explicit sentiment words are present.
Abstract
from arXiv · showhide
Recent years have brought a significant growth in the volume of research in sentiment analysis, mostly on highly subjective text types (movie or product reviews). The main difference these texts have with news articles is that their target is clearly defined and unique across the text. Following different annotation efforts and the analysis of the issues encountered, we realised that news opinion mining is different from that of other text types. We identified three subtasks that need to be addressed: definition of the target; separation of the good and bad news content from the good and bad sentiment expressed on the target; and analysis of clearly marked opinion that is expressed explicitly, not needing interpretation or the use of world knowledge. Furthermore, we distinguish three different possible views on newspaper articles - author, reader and text, which have to be addressed differently at the time of analysing sentiment. Given these definitions, we present work on mining opinions about entities in English language news, in which (a) we test the relative suitability of various sentiment dictionaries and (b) we attempt to separate positive or negative opinion from good or bad news. In the experiments described here, we tested whether or not subject domain-defining vocabulary should be ignored. Results showed that this idea is more appropriate in the context of news opinion mining and that the approaches taking this into consideration produce a better performance.
1. Introduction
News articles differ from reviews and blogs because journalists often avoid explicit sentiment and express opinions through discourse, omissions, emphasis, quotations, and framing. Document-level systems can also confuse negative news context with negative sentiment toward an entity.
- News journalists often avoid clearly positive or negative vocabulary, instead expressing opinion through discourse structure, selective facts, emphasis, and quotations.
- Document-level sentiment counting across an entire article motivated identifying each sentiment target and restricting analysis to its immediate context.
- Systems may assign negative sentiment to entities mentioned in negative news regardless of those entities’ roles in the events.
2. The EMM News Data
EMM categorises news into many subject domains using weighted or Boolean word definitions. The approach tested excludes category-defining words that overlap with sentiment vocabulary from sentiment analysis.
- EMM applications categorise news into several hundred subject domains, including disasters, security, finance, diseases, organisations, countries, and conflicts.
- EMM category definitions use Boolean expressions or weighted search-word lists with thresholds to assign domain tags.
- The method excludes category-defining words from sentiment analysis when those words belong to the domains assigned to an article.
- Existing category definitions are available for dozens of languages, although a more complete hand-produced list might be more efficient.
3. Related Work
Prior work defines opinions through topics, holders, claims, and sentiment, but newspaper annotation becomes unclear when favourability, news value, and interpretation are combined. Interpretation-dependent opinions also produced low annotator agreement.
- Opinion can be represented as a holder’s claim about a topic, optionally associated with sentiment such as good or bad.
- Sentiment analysis has encompassed favourability, support, criticism, appreciation, liking, and classifications of good versus bad news.
- Combining these aspects did not clarify what should be annotated in newspaper articles.
- Even quoted opinions showed poor agreement when interpreting them required personal convictions, background, or knowledge of the situation.
4. Experiments and evaluation
The experiments operationalized news opinion mining around explicitly stated sentiment toward identified targets, separating it from good or bad news content. Evaluation compared annotation outcomes, sentiment windows, and lexicons, with the strongest result from a combined lexicon using a six-word window.
- 4.1 Redefining the task: 1592 reported-speech quotes were selected, and restricting annotation to sentiment on the target increased inter-annotator agreement from under 50% to 60%.The final average agreement was 81% across three annotator pairs.
- 4.1 Redefining the task: The corpus marked only clear, expressly stated opinion, comprising one third of the agreed sentiment-containing quotes.The annotation framework distinguished author, reader, and text perspectives, focusing on sentiment strictly stated in the text.
- 4.2 Experiments: The system evaluated 1292 agreed quotes, successfully identifying targets in 1114, against a 61% objective-class baseline.It estimated sentiment in windows around entity mentions and removed sentiment-bearing category words associated with the article's subject domains.
- 4.2 Experiments: Smaller windows around named entities outperformed whole-text sentiment computation, while lexicons differed substantially in accuracy and in their response to category-word removal.Removing category words mattered for some resources but had no effect when lexicon entries did not overlap category vocabulary.
- 4.2 Experiments: 82% accuracy was achieved by combining JRC Tonality and MicroWN with a six-word window.Most resources did not exceed the 61% baseline, indicating that larger lexicons did not necessarily improve performance.
- 4.3 Error analysis: The main errors involved implicit or idiomatic sentiment, irony, and sentiment words referring to a different opinion target.These cases were frequently classified as neutral or assigned the wrong polarity.
5. Conclusions and future work
The paper concludes that news sentiment analysis requires clearly defined targets and sources, separation of news content from target sentiment, and explicit-opinion annotation. Experiments found that category-based separation and varied lexicon and word-window combinations produced good results when the task was clearly defined, while future work will extend the methods.
- News sentiment annotation should define the sentiment source and target before analysis.
- Analysis should separate good or bad news content from sentiment expressed toward the target.
- Annotation should focus on clearly marked, explicitly expressed opinion without requiring interpretation or world knowledge.
- Author, reader, and text views of newspaper articles require different sentiment-analysis treatment.
- Category-based separation and combinations of lexicons and word windows produced good results when the task was clearly defined.
- Future work includes negation and valence shifters, machine learning with annotated corpora, syntactic patterns, and expanded lexica.