PaperlayerView exact revision on arXiv

Source-linked AI summary

MvP: Multi-view Prompting Improves Aspect Sentiment Tuple Prediction

Zhibin Gou, Qingyan Guo, Yujiu Yang

arXiv:2305.12627v1cs.CLcs.AI

TL;DR

Fixed-order generative ABSA methods do not account for interdependent tuple elements and diverse language expressions. MvP prompts a model to generate sentiment tuples in multiple orders and aggregates the results; it reports state-of-the-art gains across 10 datasets and 4 tasks, including low-resource settings.

  • Problem

    Generative ABSA methods commonly use a fixed element order, overlooking tuple-element interdependence and language-expression diversity.

    Source: §1 · pp. 1–2

  • Method

    MvP uses element-order prompts to generate tuples from multiple views and aggregates their predictions, enabling a single model to address multiple ABSA tasks.

    Source: §1 · p. 2

  • Results

    1.34% and 1.69% absolute F1 score gains over the state of the art were reported for single-task and multi-task MvP, respectively, across all tasks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    MvP advances state-of-the-art results on 10 datasets across 4 benchmark tasks and performs effectively in low-resource settings.

    Source: §1 · p. 2, §6 · p. 9

    Training and inference overhead increase linearly with the number of views, while experiments verify improvement only on ABSA tasks.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Generative methods greatly promote aspect-based sentiment analysis via generating a sequence of sentiment elements in a specified format. However, existing studies usually predict sentiment elements in a fixed order, which ignores the effect of the interdependence of the elements in a sentiment tuple and the diversity of language expression on the results. In this work, we propose Multi-view Prompting (MvP) that aggregates sentiment elements generated in different orders, leveraging the intuition of human-like problem-solving processes from different views. Specifically, MvP introduces element order prompts to guide the language model to generate multiple sentiment tuples, each with a different element order, and then selects the most reasonable tuples by voting. MvP can naturally model multi-view and multi-task as permutations and combinations of elements, respectively, outperforming previous task-specific designed methods on multiple ABSA tasks with a single model. Extensive experiments show that MvP significantly advances the state-of-the-art performance on 10 datasets of 4 benchmark tasks, and performs quite effectively in low-resource settings. Detailed evaluation verified the effectiveness, flexibility, and cross-task transferability of MvP.

1 Introduction

Generative ABSA methods commonly predict sentiment elements in a fixed order, despite tuple elements being interdependent and language expressions diverse. MvP uses prompts to generate tuples in multiple orders and aggregates them; experiments report gains across benchmark tasks and settings.

1 Introduction

  • Fixed-order generation overlooks tuple-element interdependence and language-expression diversity, with potential incompleteness, instability, and accumulated prediction errors.Prior work typically generates sentiment elements left-to-right in a fixed order.Source: §1 · pp. 1–2
  • MvP prompts a language model to generate tuples in different element orders, then aggregates predictions from multiple views.The method is proposed to address fixed-order generation challenges.Source: §1 · p. 2, Figure 1 · p. 2
  • 1.34% and 1.69% absolute F1 score gains over the state of the art were achieved by single-task and multi-task MvP, respectively, across all tasks.Experiments cover ASQP, ACOS, ASTE, and TASD.Source: §1 · p. 2
  • Multi-task MvP trains a single model across ABSA tasks and substantially outperforms task-specific models on various tasks.The authors describe it as, to their knowledge, the first such single model.Source: §1 · p. 2
  • Experiments report state-of-the-art gains on 10 datasets across 4 tasks, with improvements also in low-resource settings.Cross-task transfer brings a more remarkable improvement than the low-resource baseline gains.Source: §1 · p. 2

2 Methodology

MvP represents aspect sentiment tuples as ordered element sequences and conditions generation on element-order prompts. It selects orders for training, generates predictions from multiple views at inference, and aggregates tuples agreed on across views.

2 Methodology

  • MvP considers every sentiment-element permutation and selects subsets for efficiency; agreement across views informs the final tuple predictions.There are 6 permutations for triplets and 24 for quadruplets; the method aggregates tuples that most views agree on.Source: §2 · p. 3

2.1 Problem Definition

  • Aspect sentiment tuple prediction maps an input sentence to all tuples of aspect term, aspect category, opinion term, and sentiment polarity.The method is presented for quadruplet tasks and can be applied to triplet tasks with minor modifications.Source: §2.1 · p. 3

2.2 Element Order-based Prompt Learning

  • MvP controls prediction order with ordered target schemas and input prompts that specify the desired sentiment-element sequence.The prompt markers are concatenated and appended to the input sentence; the target sequence follows that order.Source: §2.2 · p. 3, §2.2.2 · p. 3

2.2.1 Ordered Target Schema

  • Ordered target schemas prefix each paraphrased element with its marker and concatenate elements according to a permutation.For multiple tuples, the corresponding ordered schemas are concatenated with [SSEP].Source: §2.2.1 · p. 3

2.3.2 Training

  • MvP constructs ordered prompts and targets for each selected order, then fine-tunes a sequence-to-sequence model with negative log-likelihood.The target sequence length and previously generated tokens are used in the loss formulation.Source: §2.3 · p. 3, §2.3.2 · p. 4

2.3.1 Element Order Selection

  • For training, MvP scores candidate permutations using conditional generation scores on the training set and selects the top m orders.The candidates are all possible element permutations, ranked by average score over the training set.Source: §2.3.1 · pp. 3–4

2.4.1 Schema Constrained Generation

  • At inference, the trained model generates multiple tuples from different selected orders, with schema-constrained decoding keeping elements in their corresponding vocabularies.The schema constraint addresses outputs that do not conform to the target format, especially with small training sets.Source: §2.4 · p. 4, §2.4.1 · p. 4

2.4.2 Multi-view Results Aggregation

  • MvP aggregates predictions across views and returns tuples appearing in most views as the final output.A view may predict more than one tuple.Source: §2.4.2 · p. 4

3 Experiments

Experiments evaluate MvP on four sentiment-tuple tasks across 10 datasets, using shared model and evaluation settings and comparisons with prior approaches.

3.1 Tasks and Dataset

  • ASQP, ACOS, ASTE, and TASD are evaluated across 10 datasets, including restaurant, laptop, and cross-domain settings.The dataset collection includes implicit aspects and opinions in ACOS; the main results table covers the 10 datasets.Source: §3.1 · p. 4, Table 2 · p. 5

3.2 Implement Details

  • Experiments use T5-BASE with greedy decoding and generally five views; multi-task training combines all tasks and datasets after removing test-set overlaps.The single-task main experiment instead uses 15 views for quadruplet tasks and five for triplet tasks.Source: §3.2 · pp. 4–5

3.3 Evaluation Metrics

  • A tuple is correct only when all its elements exactly match the gold tuple; F1 is averaged over five runs with different random seeds.Multi-task runs use a different training/development split each time.Source: §3.3 · p. 5

3.4 Compared Methods

  • The evaluation compares discriminative, generative, and multi-task ABSA methods with Chat-GPT3 (gpt-3.5-turbo) zero- and few-shot prompts, and varies transfer-target training-set percentages across four tasks, datasets, and domains.DLO-top5 augments the original training set by 5 times; GPT-3.5 results are in Appendix D.Source: §3.4 · p. 5, Table 3 · p. 6

4 Results and Discussions

MvP outperforms prior methods across supervised benchmarks and also improves results in low-resource and transfer settings. The analysis examines view count, aggregation, constrained decoding, and example errors.

4.1 Single-task and Multi-task Results

  • MvP establishes new state-of-the-art results across all four tasks and 10 datasets in supervised settings.The authors report that both single-task and multi-task versions significantly outperform previous best baselines.Source: §4.1 · p. 6
  • MvP improves average F1 by 1.89% over Paraphrase, 1.34% over DLO, and 1.00 on ASTE over UIE.The UIE comparison is on ASTE datasets; the paper notes UIE uses T5-v1.1-base with further large-corpus pre-training.Source: §4.1 · p. 6
  • Multi-task MvP gains 2.82% absolute average F1 over LEGO-ABSA and remains competitive with methods requiring task-specific fine-tuning, augmentation, or complex pre-training.MvP is presented as one model for multiple ABSA tasks.Source: §4.1 · pp. 6–7

4.2 Low-resource Results

  • In low-resource settings, MvP exceeds Paraphrase and DLO by 5.70% and 3.87% average F1, respectively; cross-task transfer also improves results across triplet and quadruplet tasks.The reported TASD transfer result ranges from 50.36% to 68.78%.Source: §4.2 · p. 7

4.3 Effect Analysis

  • F1 first rises as views increase, then dips slightly beyond a point between 7 and 15 when data are adequate; more views can help in extremely low-resource settings.MvP also performs decently with a single view, and the maximum number of views is higher for quadruplets than triples.Source: §4.3 · p. 7, Figure 3 · p. 7
  • Majority voting is more stable than random selection or ranking in most cases, while constrained decoding has greater impact as training data shrink.MvP with constrained decoding performs considerably well in extremely low-resource scenarios.Source: §4.3 · pp. 7–8

4.4 Case Study & Error analysis

  • Voting filters an erroneous restaurant tuple in the Rest16 example, but multi-view predictions still confuse similar Laptop-ACOS aspect categories.Figure 4 marks positive and wrongly picked words in green and red, and ignored tuples in orange.Source: §4.4 · p. 8, Figure 4 · p. 8

5 Related Works

Related work moves from single-element ABSA prediction to joint tuple tasks and unified generative approaches. MvP is positioned as aligning element-order prompts across training and inference.

  • ABSA research progressed from predicting individual sentiment elements to joint prediction of associated pairs and then triplet and quadruplet tasks.The cited tasks include ASTE, TASD, ASQP, and ACOS, with ACOS emphasizing implicit aspects or opinions.Source: §5 · p. 8
  • Generative approaches unify ABSA tasks using sequences, natural language, or structured schemas, while multi-task methods use task prompts or instruction tuning.Prior element-order augmentation for ASQP could create multiple targets per input and discrepancies between training and inference; MvP aligns the two through multi-view prompt learning.Source: §5 · pp. 8–9

6 Conclusion

MvP aggregates predictions from different sentiment-element orders, and the paper reports strong performance across ABSA tasks in supervised and low-resource settings.

6 Conclusion

  • The multitasking model substantially outperforms task-specific models across ABSA tasks and advances the state of the art in supervised and low-resource settings.The conclusion describes results on benchmark datasets without giving specific metric values.Source: §6 · p. 9
  • MvP uses element-order-based prompt learning to aggregate results from multiple views for aspect-level opinion information prediction.Source: §6 · p. 9
  • The paper presents combining and permuting sentiment elements as a way to support multi-task modeling across ABSA tasks.Source: §6 · p. 9

Limitations

The paper identifies view-count overhead, simple aggregation, and evaluation confined to ABSA as limitations and directions for future work.

Limitations

  • Training and inference overhead grows in proportion to the number of views, though five or seven views behave decently.The authors point to a relatively small view count as a practical efficiency option.Source: Limitations · p. 9
  • The method uses a simple aggregation strategy, leaving more advanced ways to combine multiple views for future exploration.Source: Limitations · p. 9
  • Experiments verify consistent improvement only on ABSA tasks; application to other structure-prediction tasks remains an unverified possibility.The paper names information extraction, emotion-cause pair extraction, and stance detection as examples.Source: Limitations · p. 9

Ethics Statement

The authors report using established datasets, describe their analysis as fair and honest, and note a potential sentiment-polarity bias related to Internet-based pretraining.

Ethics Statement

  • The experiments use existing datasets widely used in prior public scientific papers, and the authors describe their analysis of experimental results as fair and honest.Source: Ethics Statement · p. 9
  • The authors state that the work does not harm anyone and that they open-sourced their code for further exploration.Source: Ethics Statement · p. 9
  • The authors say Internet-based pretraining may leave predicted sentiment polarity subject to unexpected bias involving gender, race, and intersectional identities.They state that this issue needs broader consideration in natural language processing.Source: Ethics Statement · p. 9

A Data Statistics

The appendix describes dataset statistics and a constrained-decoding procedure that restricts next-token candidates according to the current output state.

A Data Statistics

  • The experiments retain the train, development, and test divisions used by previous works for fair comparison.Table 9 presents statistics for datasets across ASQP, ACOS, ASTE, and TASD.Source: A Data Statistics · p. 13

B Constrained Decoding

  • Constrained decoding dynamically narrows next-token candidates to those permitted by the current state, with Table 5 listing candidate terms.For example, after an opening bracket, the candidate terms include [A], [O], [S], and [C].Source: B Constrained Decoding · p. 13, Table 5 · p. 13

C Detailed Experimental Settings

The experiments use AdamW and an Nvidia RTX 3090 GPU.

C Detailed Experimental Settings

D.1 Experiments

MvP outperforms single-view prompting across tasks, and the experiments also compare it with ChatGPT and describe prompts for ChatGPT evaluation.

D.1 Experiments

  • MvP exceeds few-shot ChatGPT by 7.06% when cross-task transferred and 22.84% when fully supervised, in absolute F1 scores.The ChatGPT evaluation tested 200 random samples per task due to budget constraints.Source: §D.1 · p. 13, Table 7 · p. 14

E Additional Comparison with Single-view Prompting

D.2 Prompts for ChatGPT

  • The paper presents zero- and few-shot prompts for ChatGPT on ASQP (R15); prompts for other tasks are available in the released code.Source: §D.2 · p. 14

E Additional Comparison with Single-view Prompting

  • Multiple views significantly improve results, especially in low-resource settings; Table 4 also shows degradation with two single-view aggregation strategies.Source: E Additional Comparison with… · p. 14
Loading 2305.12627v1…