PaperlayerView exact revision on arXiv

Source-linked AI summary

Argument Mining as a Text-to-Text Generation Task

Masayuki Kawarada, Tsutomu Hirao, Wataru Uchida, Masaaki Nagata

arXiv:2603.23949v1cs.CL

TL;DR

Argument Mining requires identifying spans, component types, and relations, while prior multi-stage systems add postprocessing and difficult hyperparameter tuning. This paper uses TANL-based text-to-text generation with pretrained encoder-decoder models to produce the annotations jointly. The approach achieves state-of-the-art performance on AAEC, AbstRCT, and CDCP, while removing irrelevant spans reduces AbstRCT inference time by approximately 30%.

  • Problem

    Prior argument-mining systems separate span identification, component classification, and relation classification, requiring complex postprocessing and difficult hyperparameter tuning.

  • Method

    The method uses TANL text-to-text generation with pretrained encoder-decoder models to jointly generate argumentative spans, components, and relations.

  • Results

    State-of-the-art Component-F1 and Relation-F1 scores were achieved across AAEC, AbstRCT, and CDCP, with irrelevant-span removal reducing AbstRCT inference time by approximately 30%.

  • Takeaways & Limitations

    TANL provides a simple text-to-text framework for predicting argumentative structures across multiple benchmark datasets.

  • Takeaways & Limitations

    Inference remains longer than previous methods and depends strongly on input length; compatibility was tested only with T5 and FLAN T5 encoder-decoder models.

Abstract

from arXiv · show

Argument Mining(AM) aims to uncover the argumentative structures within a text. Previous methods require several subtasks, such as span identification, component classification, and relation classification. Consequently, these methods need rule-based postprocessing to derive argumentative structures from the output of each subtask. This approach adds to the complexity of the model and expands the search space of the hyperparameters. To address this difficulty, we propose a simple yet strong method based on a text-to-text generation approach using a pretrained encoder-decoder language model. Our method simultaneously generates argumentatively annotated text for spans, components, and relations, eliminating the need for task-specific postprocessing and hyperparameter tuning. Furthermore, because it is a straightforward text-to-text generation method, we can easily adapt our approach to various types of argumentative structures. Experimental results demonstrate the effectiveness of our method, as it achieves state-of-the-art performance on three different types of benchmark datasets: the Argument-annotated Essays Corpus(AAEC), AbstRCT, and the Cornell eRulemaking Corpus(CDCP)

1 Introduction

Argument Mining identifies argumentative spans, component types, and relations, but prior approaches require complex subtasks, postprocessing, and difficult hyperparameter tuning. The proposed TANL-based text-to-text method generates these annotations jointly and achieves state-of-the-art results across three benchmark datasets.

  • Motivation: Argument Mining represents argumentative spans as typed nodes connected by relations in a dependency tree or directed acyclic graph.The spans contain arguments, their labels specify argument types, and edges encode relations.
  • Motivation: Earlier systems separately identify argumentative spans, classify components, and establish relations, then require postprocessing to construct valid dependency trees.These systems also need separate mechanisms and difficult hyperparameter tuning.
  • Approach: The proposed method applies TANL with a pretrained encoder-decoder to generate annotated text containing spans, components, and relations.It is presented as a simple text-to-text generation approach for argument mining.
  • Contributions: The method targets varied annotations and argumentative structures while avoiding complex postprocessing and hyperparameter tuning.The contribution is framed around architectural simplicity and adaptability across datasets.
  • Results: State-of-the-art Component-F1 and Relation-F1 scores were achieved on AAEC, AbstRCT, and CDCP using FLAN T5-XXL.The result covers three benchmark datasets and both component and relation evaluation.

2 Related Work

Related work progresses from pipeline-based argument mining to end-to-end parsing and generation approaches. The paper positions TANL text-to-text generation as a simpler alternative that uses pretrained encoder-decoder models without modifying the language model.

  • Argument Mining: Argument Mining pipelines identify spans, classify components, and classify relations, while newer systems model the structure end to end.Pipeline approaches can accumulate errors across subtasks.
  • Argument Mining: Biaffine-parser approaches treat argumentative structures as dependency trees but may require hand-crafted rules or optimum branching algorithms.These methods still use additional procedures to produce valid structures.
  • Generation Approaches: Bao et al. use an encoder-decoder with a constraint pointer mechanism, whereas this work focuses on text-to-text generation.The proposed approach uses the decoder without modifying the pretrained language model.
  • Generation Approaches: TANL extends text-to-text generation to structured prediction tasks including relation extraction, named entity recognition, semantic role labeling, and coreference resolution.Its effectiveness is associated with pretrained encoder-decoder models such as T5.

3 Proposed Methods

The method aligns argumentative annotations with input text and fine-tunes T5 using TANL to generate spans, component labels, and relations. It removes nonargumentative spans and adds representations for components with multiple parents.

  • Proposed Method: The approach aligns original text with argumentative spans, types, and relations before fine-tuning T5 with TANL.This produces annotated text for document-level argument mining.
  • Argumentatively Annotated Text: The annotation format represents dependent spans as [ ssrc | c | r = stgt ] and independent spans as [ ssrc | c ].The first form encodes a component label and relation to a target span; the second omits relation information.
  • Argumentatively Annotated Text: A generated example marks a premise and its supporting claim directly within the original input text.The output annotates both the premise span and the claim span with their relation.
  • Elimination of Unnecessary Text Spans: Nonargumentative spans are excluded to reduce the maximum token burden for document-level encoder and decoder processing.The revised format removes tokens without components or relations.
  • Multiple-Parent Components: Repeated and serial representations address components with multiple parents, which the basic TANL format cannot represent without repetition or deletion.The paper introduces these alternatives specifically for multi-parent structures in the AM datasets.

4 Experiments

Experiments evaluate the method across AAEC, AbstRCT, and CDCP using established F1 metrics, multiple pretrained model sizes, and comparisons with prior approaches. They also examine tuning, alignment, inference-time span elimination, and representations for components with multiple parents.

  • Datasets and settings: Experiments use AAEC, AbstRCT, and CDCP, with dataset-specific splits and essay-, paragraph-, or comment-level evaluation settings.AAEC includes essay- and paragraph-level tasks; AbstRCT uses the neoplasm test set; CDCP preserves 150 comments for testing.
  • Models and training: QLoRA reduces GPU memory by quantizing large models and training Low-Rank Adapters while maintaining performance comparable to full fine-tuning.The experiments apply 4-bit quantization, r = 16, α = 32, and adapters in all linear layers.
  • Models and training: Models use T5 or FLAN-T5 at four scales—Base (220M), Large (770M), XL (3B), and XXL (11B)—with results averaged across three random seeds.The study examines pretrained encoder-decoder models within the TANL framework.
  • Evaluation: Evaluation uses Component-F1 and Relation-F1, including end-to-end comparisons for AbstRCT and CDCP against prior studies using the same metrics.AAEC evaluation follows the scripts of Eger et al. (2017).
  • Multiple-parent structures: CDCP components with multiple parents require repeated or serial representations because TANL’s standard annotated-text format cannot represent such structures directly.Table 8 compares these representations on the full CDCP dataset and on data containing multiple parents.

5 Results and Discussion

The TANL text-to-text approach achieves strong argument-mining performance across benchmark datasets while simplifying model design and reducing AbstRCT inference time. Larger models improve results, especially for long-range relations, and serial representations benefit relation classification.

  • FLAN T5-XXL exceeded existing top F1 scores on AAEC, with Component-F1 scores of 80.15 essay-level and 78.40 paragraph-level, and Relation-F1 scores of 61.19 and 61.87.
  • The approach uses a straightforward architecture with QLoRA, while relying only on the learning rate as its hyperparameter instead of separate subtask-specific tuning.
  • FLAN T5-XXL achieved state-of-the-art Component-F1 and Relation-F1 scores of 72.86 and 47.66 on AbstRCT, and 72.68 and 33.96 on CDCP.
  • Removing irrelevant text spans preserved performance across three evaluated tasks and reduced AbstRCT inference time by approximately 30% for Base, Large, XL, and XXL models.
  • Serial representations consistently outperform repeated representations in relation classification, although component-classification performance shows no significant difference.
  • Increasing parameters particularly improves long-distance dependency detection, including the MAJORCLAIM-CLAIM relation, which rose from 45.33 to 57.14.

6 Conclusion

The paper concludes that TANL-based text-to-text generation is a simple yet strong approach for argument mining. It outperformed current state-of-the-art methods on three datasets, and removing irrelevant spans reduced AbstRCT inference time by approximately 30%.

  • The TANL framework provides a simple text-to-text generation approach for argument mining.
  • The approach outperformed current state-of-the-art methods on AAEC, AbstRCT, and CDCP.
  • Removing irrelevant text spans decreased AbstRCT inference time by approximately 30%.

Limitations

The method’s practical use is constrained by inference-time costs, hardware requirements for large models, and limited validation beyond encoder-decoder architectures.

  • Inference time depends strongly on input length and remains longer than previous methods despite removing irrelevant spans.The authors identify inference time as a significant hurdle for practical implementation.
  • Large-parameter models still require GPUs with substantial memory capacity, such as the A100 with 80GB.QLoRA reduces training memory requirements but does not eliminate the hardware constraint.
  • Experiments used TANL only with encoder-decoder models such as T5 and FLAN T5.Compatibility with decoder-based models remains unverified.
  • A preliminary GPT-4 Turbo few-shot experiment produced unsatisfactory results.The authors provide this as an initial indication rather than a comprehensive decoder-based evaluation.

A Implementation Details

Implementation used TANL and QLoRA scripts with T5 and FLAN T5 models, while hyperparameters were tuned separately for AAEC and the other datasets.

  • A Implementation Details: The appendix provides separate hyperparameter tables for AAEC, AbstRCT and CDCP, and QLoRA fine-tuning.
  • A Implementation Details: The implementation was based on TANL and QLoRA scripts and used T5 and FLAN T5 models from Hugging Face.
  • A Implementation Details: Hyperparameters were tuned on AAEC at the essay level.
  • A Implementation Details: Learning rates for Base, Large, XL, and XXL models were 0.0005, 0.0005, 0.0002, and 0.0002, respectively, for both T5 and FLAN T5.The values were selected by testing increments from 0.0001 to 0.0005.

B Decoder-based Large Language Models

The decoder-based evaluation used GPT-4 Turbo few-shot prompting, whose scores remained below fine-tuned FLAN T5; the authors therefore leave broader decoder integration for future work.

  • B Decoder-based Large Language Models: GPT-4 Turbo was evaluated with 3-shot, 5-shot, 10-shot, and 20-shot learning.The experiment used the gpt-4-1106-preview version of the OpenAI API.
  • B Decoder-based Large Language Models: 55.51 Component-F1 and 28.38 Relation-F1 were obtained with 20-shot learning, versus 48.99 and 24.78 with 3-shot learning.
  • B Decoder-based Large Language Models: GPT-4 Turbo few-shot scores were notably lower than those achieved by the fine-tuned FLAN T5 model.
  • B Decoder-based Large Language Models: The study did not extensively investigate LLM applications in Argument Mining because effective fine-tuning prompts remain challenging.The authors suggest that the text-to-text framework could nevertheless be integrated with decoder models.
Loading 2603.23949v1…