InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis

1 Introduction

• InstructABSA instruction-tunes Tk-Instruct for six ABSA subtasks using task-specific prompts with positive, negative, and neutral examples.The prompts provide task definitions followed by examples.Source: §1 · p. 1
• 5.69% points on Rest14 ATE and 9.59% points on Rest15 ATSC: InstructABSA surpasses SOTA; it also beats SOTA by 3.37% points on Lapt14 ASPE.For ASPE, it also surpasses SOTA by 1.4% points on Rest14.Source: §1 · pp. 1–2
• Roughly 20% of training samples yields competitive scores against Varia et al. (2023), while delusive examples reduce results by about 10%.The introduction also reports a sizable performance gap of approximately 20% between InstructABSA and LoRA methods.Source: §1 · p. 2
• The 200M model outperforms or matches prior SOTA models with 1.5B parameters, with evidence of cross-domain and joint-domain generalization.The introduction also reports competitive results on AOOE and AOSTE.Source: §1 · p. 2

2 InstructABSA: Instruction Learning for ABSA

• ABSA inputs are review sentences with aspect terms, corresponding opinion terms, and sentiment polarities labeled positive, negative, or neutral.The formulation treats aspect terms and their associated opinions and polarities as sentence-level task elements.Source: §2 · p. 2
• The supplied captions identify ATE results by F1 score and ATSC results by accuracy.The captions name prior systems whose results appear in the respective tables.Source: Table 1 · p. 2, Table 2 · p. 2
• Instruction tuning prepends task-specific prompts to each subtask input to produce an instruction-tuned model.The section refers to further details in §G.Source: §2 · p. 2

3.1 Sub Task Results

• InstructABSA surpasses SOTA on ATE across Lapt14, Rest14, Rest15, and Rest16, on Rest15 ATSC, and on all four ASPE datasets; it also outperforms IOG and ONG on Lapt14 AOOE.The ASPE results are reported as F1 scores in Table 3.Source: §3.1 · p. 2, Table 3 · p. 3
• In AOPE, InstructABSA falls short of existing models, while Seq2Path has the highest AOSTE F1 scores; InstructABSA remains competitive on Rest14 AOSTE.The authors attribute weaker AOPE and AOSTE performance to exposure bias when opinion terms are not grounded and may be any vocabulary word.Source: §3.1 · pp. 2–3

3.2 Analysis

• In cross-domain evaluation, InstructABSA-2 scores 80.56 ATSC accuracy versus 75.53 for InstructABSA-1 when trained on Lapt14 and tested on Rest14; joint-domain ATE surpasses prior SOTA.Joint-domain training combines laptop and restaurant data and evaluates on both test sets.Source: §3.2 · p. 3, Table 8 · p. 3
• InstructABSA-1 and -2 outperform vanilla models across most ABSA subtasks, while the instruction analysis compares task definitions and example configurations, including delusive examples.Figure 2 distinguishes vanilla T5, instruction-tuned T5, positive-example configurations, and a delusive configuration with incorrect input-output mappings.Source: §3.2 · pp. 3–4, Figure 2 · p. 4
• With roughly 50% of training samples, InstructABSA achieves competitive scores with its best results, and vanilla T5 performs consistently worse at matched sample sizes.Figure 3 compares sample sizes as percentages of training data.Source: §3.2 · p. 4, Figure 3 · p. 4
• Compared with InstructABSA, LoRA has score drops of 13.32 points on ATE, 26.8 on ATSC, and 19.8 on ASPE.The authors note LoRA can improve memory and computational efficiency, but characterize these performance drops as significant.Source: §3.2 · p. 4, Table 9 · p. 4

• InstructABSA is an instruction-tuned modeling approach for all ABSA subtasks.Source: §4 · p. 4
• The approach surpasses previous scores on several tasks and achieves competitive scores on the rest using a significantly smaller model than previous approaches.Source: §4 · p. 4
• The authors report analyzing the approach’s performance along several lines of enquiry and releasing the code.Source: §4 · p. 4

• The study is limited to the Sem Eval 2014, 15, and 16 datasets.Source: Limitations · p. 5
• Experiments use a 200M model, which the authors say may limit applicability to smaller models.Source: Limitations · p. 5
• The study uses English Tk-Instruct models, so the authors caution that findings may not directly apply to other languages.Source: Limitations · p. 5

• The T5 model used in the experiments may have biases from its pretraining and instruction-tuning data.Source: Ethical Considerations · p. 5
• The authors did not conduct stress testing.Source: Ethical Considerations · p. 5
• The authors state that they believe the research raises no additional issues related to privacy, fairness, bias, or discrimination.Source: Ethical Considerations · p. 5

A Choosing Samples as Instruction Exemplars:

• Across Lapt14, Rest14, Rest15, and Rest16, aspect counts centered on one, two, and three aspects, representing 30%, 11%, and 4.5% of total aspects, respectively.Source: A Choosing Samples as Instruction… · p. 8
• Instruction exemplars are randomly selected from samples with one to three aspects, and those exemplars are excluded during evaluation.Source: A Choosing Samples as Instruction… · p. 8

B Instruction Effectiveness Study

• For most ABSA subtasks, InstructABSA-1 and InstructABSA-2 instruction configurations yield the best performance.Source: B Instruction Effectiveness Study · p. 8
• Task definitions are tested as original, simplified, and empty; the empty configuration is tested only with vanilla T5 and vanilla Tk-Instruct.Source: B Instruction Effectiveness Study · p. 8
• Task-example conditions include original positive examples, delusive negative examples with incorrect mappings, and an empty condition without examples.Source: B Instruction Effectiveness Study · p. 8
• Additional example configurations compare two positive, negative, or neutral examples and one example of each polarity.Source: B Instruction Effectiveness Study · p. 8
• Vanilla T5 and Tk-Instruct models do not give the best results, which the authors interpret as supporting further instruction tuning.Source: B Instruction Effectiveness Study · p. 8

C Detailed Dataset Description:

• 1,557 Lapt14 and 1,020 Rest14 training reviews have no aspect terms or corresponding polarities.The test set also includes 378 such Lapt14 reviews and 194 such Rest14 reviews.Source: C Detailed Dataset Description · p. 8
• ATSC dataset statistics are presented in Table 10, and conflict labels are ignored to remain consistent with previous approaches.The table caption defines Pos., Neg., and Neut. as positive, negative, and neutral.Source: C Detailed Dataset Description · p. 8, Table 10 · p. 8

D Experimental Setup

• InstructABSA uses Tk-Instruct-base-def-pos as its instruction-tuned model, with prompts defining ABSA subtasks and examples.InstructABSA-1 uses two positive examples; InstructABSA-2 uses two each of positive, negative, and neutral examples.Source: D Experimental Setup · p. 8
• Experiments use SemEval 2014, 2015, and 2016 customer reviews from laptop, hotel, and restaurant domains.The datasets include Lapt14, Hotel15, Rest14, Rest15, and Rest16.Source: D Experimental Setup · p. 8
• Figure 4 compares instruction configurations on ATE, AOSTE, and ATSC, including task-definition-only, example-based, and incorrectly mapped prompts.The caption identifies vanilla models trained without instructions and five example configurations.Source: Figure 4 · p. 9
• Evaluation uses F1-score for ATE, AOPE, AOOE, and AOSTE, and accuracy for ATSC.The stated metrics follow previous approaches (Luo et al., 2020).Source: D Experimental Setup · p. 9

E Extended Related Work

• Instruction-learning studies cited here address cross-task generalization (Mishra et al., 2022b), zero-shot generalization (Wei et al., 2022), and task instruction reframing (Mishra et al., 2022a).The related work also cites instruction use in biomedical multi-task settings (Parmar et al., 2022) and adding knowledge to instructions (Gupta et al., 2022; Gupta et al., 2021).Source: E Extended Related Work · pp. 9–10

F Additional Tables for Plots

• The analysis spans ATE, ATSC, and AOSTE, chosen to represent increasing task difficulty from term extraction to sentiment association and triplet extraction.The passage describes ATE as easiest, ATSC as more complex because it links aspect terms with sentiment polarity, and triplet extraction as most difficult.Source: F Additional Tables for Plots · p. 10
• Tables 12–14 report F1 scores for instruction configurations on ATE, ATSC, and AOSTE, while Table 15 provides raw sample-efficiency values.The passages identify these as unaggregated values underlying the instruction-effectiveness and sample-efficiency plots.Source: F Additional Tables for Plots · p. 10, Table 12 · p. 10, Table 13 · p. 10

G InstructABSA prompt examples

• For ATSC, an instruction prompt and sentence are input, and the output is the sentiment polarity for the corresponding aspect.The prompt examples include InstructABSA-2 formats for ATE, ATSC, and AOPE.Source: Figure 5 · p. 10, G InstructABSA prompt examples · p. 10
• The appendix illustrates AOPE prompting with task definition only, two negative exemplars, two neutral exemplars, one positive, one negative and one neutral exemplar, and delusive instructions with output labels incorrectly mapped to inputs.Source: Table 22 · p. 15, Table 23 · p. 15, Table 24 · p. 15