Source-linked AI summary
Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised
Stefanos Angelidis, Mirella Lapata
TL;DR
Opinion summarization must condense large collections of product reviews while identifying which aspects and sentiments make opinions salient. The paper combines weakly supervised aspect extraction and multiple-instance sentiment prediction into an extractive framework, introduces a six-domain dataset, and reports improvements over baselines with human preference for its summaries.
Problem
Consumers and manufacturers need concise access to large volumes of opinionated review text, but opinion summarization requires identifying aspects, sentiment, and useful summary content.
Method
A neural framework combines a multi-task seeded aspect extractor with a multiple-instance-learning sentiment predictor and greedy redundancy-aware extractive selection using weak supervision.
Results
The approach delivers significant improvements over strong baselines across aspect identification, salient opinion extraction, and summary generation, while human judges prefer its summaries.
Takeaways & Limitations
Weak supervision can support aspect-based opinion summarization across multiple subtasks, with summaries favored over competitive extractive and abstractive systems in human judgments.
Takeaways & Limitations
The authors identify joint aspect-sentiment modeling, additional languages and domains, and weakly supervised abstractive summarization as future work.
Abstract
from arXiv · showhide
We present a neural framework for opinion summarization from online product reviews which is knowledge-lean and only requires light supervision (e.g., in the form of product domain labels and user-provided ratings). Our method combines two weakly supervised components to identify salient opinions and form extractive summaries from multiple reviews: an aspect extractor trained under a multi-task objective, and a sentiment predictor based on multiple instance learning. We introduce an opinion summarization dataset that includes a training set of product reviews from six diverse domains and human-annotated development and test sets with gold standard aspect annotations, salience labels, and opinion summaries. Automatic evaluation shows significant improvements over baselines, and a large-scale study indicates that our opinion summaries are preferred by human judges according to multiple criteria.
1 Introduction
Opinion summarization helps users and manufacturers process large volumes of online opinionated text. This paper introduces a weakly supervised neural framework that combines aspect extraction, sentiment prediction, and extractive summary generation.
- Motivation: Opinion summarization aggregates opinions from online reviews and related sources to support information access.It can help consumers make informed decisions and manufacturers monitor customer views.
- Task decomposition: Entity-centric opinion summarization commonly decomposes into aspect extraction, sentiment prediction, and summary generation.These subtasks identify entity features and their expressions, determine polarity, and present selected opinions.
- Approach: The paper presents a neural framework that uses product domains, user ratings, and minimal aspect keywords as weak supervision.It combines an aspect discovery model with a weakly supervised sentiment predictor and greedily produces extractive summaries while avoiding gold salience labels and summaries during training.
- Resources: The introduced dataset covers Amazon reviews from six product domains with annotated development and test sets.Annotations include aspects, salience labels, and multi-document extractive summaries.
- Results: Automatic evaluations and a large-scale user study show stronger extraction and summary quality than comparison systems.Human judges preferred the resulting summaries across multiple criteria.
2 Related Work
Prior opinion-summarization research addressed aspect extraction, sentiment prediction, and multi-document summarization through largely separate methods. The paper positions its work as a neural, weakly supervised framework integrating these components for end-to-end opinion summarization.
- Opinion summarization: Earlier opinion-summarization literature includes extensive non-neural methods, while neural end-to-end systems without direct supervision were not previously reported by the authors.Recent work instead tackled individual subtasks independently.
- Aspect extraction: Aspect extraction has used linguistic analysis, clustering, data mining, information extraction, supervised sequence labeling, and unsupervised neural models.The cited approaches identify product features or aspect terms using varied representations and objectives.
- Sentiment prediction: Sentiment prediction has combined lexicons and rules with supervised neural models and multiple instance learning from review ratings.MIL methods train segment- or sentence-level predictors using freely available overall ratings.
- Multi-document summarization: Neural multi-document summarization has used ranking, multi-task learning, and graph-based sentence-relation modeling for extractive selection.The paper adapts the multi-task idea to opinion summarization.
3 Problem Formulation
The paper formulates summarization as selecting a small subset of salient opinion tuples from product reviews. Each tuple links a review segment to its discussed aspects and conveyed polarity, with aspect-specific comments favored as useful reasons for overall ratings.
- Corpus and segmentation: A corpus contains products from a domain, each with reviews accompanied by overall author ratings and segmented into sentences, phrases, or EDUs.EDUs are clause-like units obtained with a rhetorical-structure parser.
- Aspects: Each segment may discuss multiple domain aspects, including specific attributes such as television picture quality, sound quality, and connectivity.Segments without a specific attribute are assigned a general aspect.
- Opinion representation: An opinion is represented as os = (s, A_s, pol_s), combining a segment, its discussed aspects, and polarity from −1 to +1.The polarity scale runs from maximally negative to maximally positive sentiment.
- Selection objective: For each product, the goal is to select a small subset of opinions that forms a summary of the most salient review content.The formulation favors aspect-related comments because they provide specific reasons for customers’ overall opinions, while ratings capture general experience.
- Summary construction: The proposed summarization approach combines aspect, sentiment, and redundancy information to produce opinion summaries.The paper describes the approach in separate aspect-extraction and summary-generation stages.
4 Aspect Extraction
The paper extends unsupervised aspect discovery into a weakly supervised extractor that uses seed words and product-domain signals. It combines reconstruction, domain classification, and attention-based encoding to identify aspect-specific review segments with less manual mapping.
- Aspect-Based Autoencoder: ABAE learns segment-level aspect predictors by reconstructing segment encodings as combinations of aspect embeddings, without direct aspect supervision.It uses word and aspect embedding dictionaries, attention-based segment encoding, softmax aspect prediction, and a reconstruction loss with sampled negative segments.
- Aspect-Based Autoencoder: ABAE discovers fine-grained topics without predefined aspects, requiring manual interpretation and many-to-one mapping to genuine coarse-grained aspects.The model may use substantially more discovered topics than actual aspects, creating an additional post-hoc mapping burden.
- Multi-Seed Aspect Extractor: MATE introduces weakly supervised aspect extraction using small sets of seed words supplied by experts or selected from aspect-annotated reviews.Seeds act as query terms for segments discussing each aspect and replace ABAE’s single aspect dictionary with multiple seed matrices.
- Multi-Seed Aspect Extractor: MATE reduces each seed matrix to an aspect embedding using seed weights, which may be uniform, fixed, learned, or dynamically assigned.The reported experiments focus on fixed weights because they produced the most stable performance across domains.
- Multi-Task Objective: The multi-task extension trains the encoder to support both aspect-based reconstruction and product-domain classification using reviews from multiple corpora.Reconstruction uses only in-domain segments, while the domain-classification loss uses all segments; out-of-domain reconstruction is avoided because domain aspect mismatches can confuse extraction.
5 Opinion Summarization
The framework combines aspect predictions, weakly supervised segment-level sentiment, and redundancy-aware selection to produce extractive opinion summaries.
- The summarization framework combines aspect extraction, polarity prediction, and segment selection to identify and retain salient opinions.Its pipeline uses aspect and sentiment information before discarding redundant opinions.
- Opinion Polarity: MILNET learns segment-level sentiment predictions from document-level user ratings using a hierarchical, attention-based neural architecture.Review segments are encoded, classified, and aggregated into document-level predictions during end-to-end training.
- Opinion Ranking: Each opinion is represented by its segment, highest-probability aspect, and polarity, then ranked by a salience score.The score favors strongly positive or negative opinions that are likely to discuss a non-general aspect.
- Opinion Ranking: The salience score combines polarity magnitude with the probability difference between the most probable aspect and the general aspect.The system currently assigns each segment only its highest-probability aspect, though multiple aspects could be allowed.
- Opinion Selection: A greedy selection algorithm starts with the highest-ranked opinion and rejects candidates whose cosine similarity with any selected segment is at least 0.5.This removes paraphrastic redundancy from the final extractive summary.
6 The OPOSUM Dataset
OPOSUM is a six-domain Amazon review dataset for training and evaluating opinion summarization, with human-annotated development and test subsets.
- OPOSUM contains Amazon reviews from six product domains: Laptop Bags, Bluetooth Headsets, Boots, Keyboards, Televisions, and Vacuums.The training collections were downsampled from the Amazon Product Dataset and include reviews with ratings.
- The reviews were segmented into elementary discourse units using a publicly available RST parser.
- The human-annotated subset contains 600 reviews, split evenly between development and test sets, with 10 products and 10 reviews per product in each domain.Products were sampled uniformly across ratings.
- Annotations include EDU-level aspects, salience labels, and gold-standard multi-document extractive opinion summaries.Aspect labels were collected from three annotators with majority-vote aggregation.
- Aspect annotation agreement across domains was Cohen’s Kappa K = 0.61 over 8,175 annotated segments and three annotators.
- System evaluation uses automatic measures, including ROUGE for agreement with reference summaries, alongside human annotation of salient opinions.The benchmark evaluates both identification of important opinions and summary agreement.
7 Experiments
Experiments evaluate aspect identification, salient opinion extraction, and summary generation using OPOSUM and multiple automatic and human-evaluation protocols. The proposed models outperform comparison systems and are preferred by crowdworkers across all human-evaluation criteria.
- The experiments assess aspect identification, salient opinion extraction, and summary generation.
- Aspect Extraction: MATE improves micro-averaged F1 by 3.2%, while multi-task training adds a further 2.7%.The models outperform both ABAE variants across domains.
- Opinion Salience: Combining polarity and aspect information improves salient-opinion retrieval across domains.MILNET+MATE+MT achieves a 2.6% MAP increase over MILNET+MATE and 4.6% over MILNET+ABAEinit.
- Opinion Summaries: MILNET+MATE+MT significantly outperforms all comparison systems on ROUGE-1, ROUGE-2, and ROUGE-L F1.A redundancy filter slightly improves performance, while aspect predictions benefit Opinosis without closing the gap.
- Human Evaluation: Crowdworkers favored the proposed model over comparison systems on informativeness, polarity, coherence, and redundancy.All differences were statistically significant at p < 0.05; human summaries were significantly preferred only for coherence.
- Qualitative Analysis: In an example, the proposed summary largely captures aspect-relevant opinions, while its redundancy filter misses some similar picture-quality opinions.LexRank selects many general or non-opinionated comments, and Opinosis captures aspects but produces grammatical errors.
8 Conclusions
The paper presents a weakly supervised neural framework for aspect-based opinion summarization and evaluates it across three subtasks. It reports improvements over strong baselines and human preference over competing extractive and abstractive systems, while identifying broader integration and coverage as future directions.
- The framework combines a seeded multi-task aspect extractor with a multiple-instance-learning sentiment predictor.
- The approach improves over strong baselines across aspect identification, salient opinion extraction, and summary generation.
- Crowdworkers favor the summarizer over competitive extractive and abstractive systems.
- Future work targets jointly identifying aspects and sentiment, expanding to languages and domains, and developing weakly supervised abstractive summarization.