Source-linked AI summary
Multiple Instance Learning Networks for Fine-Grained Sentiment Analysis
Stefanos Angelidis, Mirella Lapata
TL;DR
Fine-grained sentiment analysis aims to identify opinions within documents, but segment-level sentiment labels are costly to obtain. The paper trains MILNET from document labels to predict segment polarities and extract opinion snippets. It reports strong segment-classification performance and participant preference for MILNET’s opinion extracts.
Problem
Fine-grained sentiment analysis needs segment-level sentiment labels, whose acquisition is laborious and expensive despite their relevance to opinion mining.
Source: §1 · p. 1Method
MILNET learns segment sentiment from document-level labels, then uses attention-weighted polarity scores to identify positive and negative snippets.
Source: §1 · p. 1, §3.2 · p. 5, §4 · p. 6Results
MILNET with gated, segment-specific polarities performs best among document-supervised models across four datasets and comparably to fully supervised Seg-CNN; participants preferred its opinion extracts.
Source: §6.1 · p. 10, §7 · p. 12Takeaways & Limitations
The model’s segment predictions can be used to produce extractive sentiment summaries from unseen texts.
Source: §4 · p. 6
Abstract
from arXiv · showhide
We consider the task of fine-grained sentiment analysis from the perspective of multiple instance learning (MIL). Our neural model is trained on document sentiment labels, and learns to predict the sentiment of text segments, i.e. sentences or elementary discourse units (EDUs), without segment-level supervision. We introduce an attention-based polarity scoring method for identifying positive and negative text snippets and a new dataset which we call SPOT (as shorthand for Segment-level POlariTy annotations) for evaluating MIL-style sentiment models like ours. Experimental results demonstrate superior performance against multiple baselines, whereas a judgement elicitation study shows that EDU-level opinion extraction produces more informative summaries than sentence-based alternatives.
1 Introduction
Fine-grained sentiment analysis can reveal positive and negative opinions within a review whose overall rating is negative. The paper uses document-level supervision to predict segment sentiment and introduces SPOT to evaluate such predictions.
1 Introduction
- A review’s overall rating can obscure positive and negative experiences that appear in individual snippets.The example is a 2-star restaurant review with snippets of opposing polarities.Source: §1 · p. 1, Figure 1 · p. 1
- MILNET learns segment polarities from document-level labels, avoiding the costly acquisition of segment-level sentiment annotations.The paper also introduces SPOT, a dataset of sentence- and EDU-level polarity annotations for evaluation.Source: §1 · pp. 1–2
- The model represents documents as sentences or as clauses treated as Elementary Discourse Units (EDUs).A discourse parser identifies the EDUs.Source: §1 · p. 2
- The paper presents MILNET, SPOT, and evaluations reporting stronger segment-sentiment detection and more informative opinion extraction than baselines.The evaluation combines automatic and human-based methods.Source: §1 · p. 2
2 Background
Prior work spans supervised sentiment classifiers, hierarchical document models, and multiple instance learning. This paper positions segment-polarity prediction and opinion extraction as uses of document-level supervision, including evaluation of extraction quality.
Sentiment classification
- Neural sentiment work includes Kim’s CNN for sentence classification and recursive models that learn sentiment at varying text granularities.The paper describes Kim’s approach further because it also uses it as part of its model.Source: §2 · p. 2
Neural sentiment models
- The background describes CNN segment encoding through word embeddings, windowed convolution, feature maps, and max-overtime pooling.The resulting fixed-size segment representation is used for sentiment prediction.Source: §2 · p. 2
Hierarchical document models
- Hierarchical classifiers build document representations from sentence representations, with attention models weighting text parts by context.The paper uses a hierarchical attention network as a point of comparison.Source: §2 · pp. 2–3
Opinion mining
- The paper distinguishes its approach by predicting polarity for individual segments and using attention to support opinion detection, rather than only aggregating sentence representations.It applies this to individual reviews and trains without fine-grained labels or gold-standard opinion summaries.Source: §2 · p. 3
Multiple instance learning
- MIL assigns labels to bags of unlabeled instances and aggregates instance predictions; prior approaches include averaging and weighted summation (Xu and Frank, 2004; Pappas and Popescu-Belis, 2014; 2017).The paper treats segment sentiment as a setting where both bag and instance labels are inferred.Source: §2 · pp. 3–4
- Sentiment-focused MIL work includes latent sentence labels, aspect sentiment regression, and averaged sentence predictions (Täckström and McDonald, 2011; Pappas and Popescu-Belis, 2014; Kotzias et al., 2015).The cited Kotzias et al. approach uses pretrained sentence embeddings and is not trainable end-to-end.Source: §2 · p. 4
3 Methodology
The methodology formulates segment-level sentiment as multiple instance learning: documents have labels, while the segment labels are unobserved. MILNET predicts segment distributions and uses attention to combine them into document sentiment.
3.1 Problem Formulation
- In the MIL formulation, each labeled document is a bag of segments with unobserved labels, and its sentiment label is an unknown function of those segment labels.The ordered document label set runs from maximally negative to maximally positive.Source: §3.1 · p. 4
3.2 Multiple Instance Learning Network
- MILNET uses a CNN segment encoder, a softmax segment classifier, and attention-based weighting of segment predictions.It learns from document-level sentiment labels while predicting segment polarities.Source: §3.2 · p. 5
- Simple averaging of segment class distributions is a possible document prediction, but the method instead weights segments because their sentiment clues differ in importance.The model is trained end-to-end on documents with user-generated sentiment labels using document-level prediction loss.Source: §3.2 · pp. 5–6
- The model replaces averaging segment vectors with attention-weighted aggregation of segment sentiment predictions.The attention mechanism is inspired by Yang et al. (2016), but aggregates predictions rather than segment representations.Source: §3.2 · pp. 5–6
- A bidirectional GRU provides segment representations for attention, whose learned key is intended to recognize sentiment-heavy segments.The segment-specific attention weights are normalized similarities to that learned key.Source: §3.2 · p. 6
4 Polarity-based Opinion Extraction
After training, the model predicts sentiment distributions for unseen segments, enabling extractive opinion summaries. Polarity scoring ranks segments, while attention-gated scores and EDU segmentation help distinguish sentiment-bearing snippets.
4 Polarity-based Opinion Extraction
- The trained model predicts sentiment distributions for unseen segments, from which highly positive and negative snippets can be selected as extractive summaries.The method is presented as a direct application of segment-level prediction.Source: §4 · p. 6
Polarity Scoring
- Each segment’s class-probability distribution is converted to a real-valued polarity score by taking its dot product with uniformly spaced weights over the ordered labels.For five classes, the weights range from −1 to 1 in 0.5 increments.Source: §4 · p. 6
Gated Polarity
- The gated extension multiplies segment polarity by its attention weight, moving scores for unattended segments closer to zero.This further distinguishes segments with significant sentiment cues from those without them.Source: §4 · pp. 6–7
Segmentation Policies
- The method scores either sentences or RST-parser-derived EDUs, using RST segmentation but not its tree structure.EDUs correspond roughly to independent clauses; potential use of the tree structure is left for future work.Source: §4 · p. 7
- In the example, one mixed-sentiment sentence contains EDUs that clearly convey opposing sentiments.The example illustrates why EDU-based segmentation may benefit opinion extraction.Source: §4 · p. 7
5 Experimental Setup
The setup trains on Yelp’13 and IMDB document ratings and evaluates segment-level predictions using SPOT, annotated at sentence and EDU granularity. It compares MILNET with alternative systems and variants, using cross-validated macro-averaged F1 for segment classification.
5.1 Datasets
- SPOT is a segment-level sentiment dataset built from sampled Yelp’13 and IMDB reviews, with document-level classes uniformly represented and lengths representative of each corpus; reviews were segmented into sentences and EDUs.Source: §5.1 · p. 7
- Positive- and negative-labeled documents contain more segments of the corresponding polarity, while neutral EDUs are more common than neutral sentences.The authors suggest filtering neutral EDUs could yield more concise opinion extraction than using whole sentences.Source: §5.1 · p. 8, Figure 4 · p. 8
5.2 Model Comparison
- MILNET naturally assigns distinct polarities to segments, whereas HIERNET assigns one polarity score across segments; gating can identify neutral segments in either model.The comparison also includes majority-class, SO-CAL, fully supervised Seg-CNN, and GICF, an unweighted-average MIL method using sentence features from a pretrained CNN.Source: §5.2 · p. 8
5.3 Model Training and Evaluation
- The models were trained for 25 epochs with Adadelta, 300-dimensional word2vec embeddings, and batches of 200 documents, with padding minimized by length-based batching.CNN window sizes were 3, 4, and 5 with 100 feature maps each; evaluation used macro-averaged F1 and thresholds selected by 10-fold cross-validation. Seg-CNN is not directly comparable because it uses segment labels and 1K–2K segment labels versus about 250K document labels.Source: §5.3 · pp. 8–9
6 Results
The results evaluate segment classification and opinion extraction, finding that document-supervised MILNET performs strongly across datasets and that EDU-based extracts are often preferred for informativeness and polarity.
6.1 Segment Classification
- MILNET with gated, segment-specific polarities achieves the best classification performance among document-supervised models across all four datasets, comparable to fully supervised Seg-CNN.It beats SO-CAL on all but one dataset; gated polarities help all model configurations, while document-level polarity predictions perform worse overall.Source: Table 3 · p. 10, §6.1 · p. 10
- Gating moves neutral-segment polarity scores toward zero; gated variants improve neutral identification, and MILNET remains more effective overall across the four SPOT datasets.Figure 6 shows MILNET’s neutral scores concentrated near zero, unlike HIERNET’s wider spread; the reported neutral-class F1 comparison is in Table 4.Source: §6.1 · p. 10, Table 4 · p. 10, Figure 6 · p. 10
- With a few thousand training documents, MILNET beats supervised Seg-CNN except on IMDB with EDU segmentation, where low-quality EDUs may contribute to weaker performance.The training-size results average five random subsets at each size; Figure 7 presents the comparison.Source: §6.1 · p. 11, Figure 7 · p. 11
- On the Yelp and IMDB sentence datasets, MILNET outperforms all GICF variants, and its sentence embeddings improve GICF’s performance.Table 5 reports accuracy on the sentence-classification datasets introduced by Kotzias et al. (2015).Source: Table 5 · p. 10, §6.1 · p. 11
6.2 Opinion Extraction
- In human judgments, MILNET is preferred to HIERNET for informativeness and polarity, while EDU-based MILNET summaries beat its sentence-based summaries on those criteria but not coherence.HIERNET is more often preferred for coherence against MILNET, but that difference is not statistically significant; participants judged summaries at a 30% compression rate on informativeness, polarity, and coherence.Source: §6.2 · pp. 11–12, Table 6 · p. 11
- MILNET summaries are also judged more informative and better at distilling polarity than LEAD and RANDOM baselines; EDU segmentation yields more concise grouping of positive and negative snippets.Figure 8 illustrates the summaries; unlike HIERNET’s document-level polarity scores, MILNET assigns individual segment polarities that can identify positive and negative snippets.Source: §6.2 · p. 12, Figure 8 · p. 12
7 Conclusions
The conclusion presents MILNET as a document-supervised neural model for fine-grained sentiment analysis, with positive human-evaluation results for its opinion extracts, especially with EDU segments.
7 Conclusions
- MILNET predicts sentiment for sentence- or EDU-level segments and combines those predictions up the document hierarchy without segment-level labels.The conclusion frames this as multiple instance learning that can use large-scale sentiment classification datasets.Source: §7 · p. 12
- Attention-weighted polarity scoring provides a way to extract sentiment-heavy opinions from the predicted segment polarities.Source: §7 · p. 12
- Experiments report superior performance to more conventional neural architectures, and human evaluation finds MILNET extracts effective for informativeness and polarity, especially with EDU segments.Future work named in the conclusion includes multi-document, aspect-based extraction and improving summary coherence with finer-grained discourse information.Source: §7 · p. 12