Source-linked AI summary
Neural End-to-End Learning for Computational Argumentation Mining
Steffen Eger, Johannes Daxenberger, Iryna Gurevych
TL;DR
The paper addresses end-to-end computational argumentation mining without relying on manually engineered pipeline systems. It evaluates token-based dependency parsing, sequence tagging, and multi-task learning, finding that BiLSTM tagging is robust while dependency parsing performs poorly and auxiliary-task learning improves performance.
Problem
End-to-end AM requires identifying and classifying components and relations, but pipeline systems depend on manually adapted features and constraints and can propagate errors between subtasks.
Method
The paper evaluates token-based dependency parsing, distance-aware sequence tagging, multi-task tagging, and a model combining sequential and tree information.
Results
BiLSTM taggers perform robustly across classification settings, token-based dependency parsing is ineffective, and multi-task learning with natural auxiliary subtasks improves performance.
Takeaways & Limitations
AM is better supported by local neural tagging that models component and relation tasks jointly than by a token-level tree constraint or naive coupling.
Takeaways & Limitations
Exact argument-span segmentation remains difficult, although neural taggers identify boundaries better than the ILP model and neural parsers.
Abstract
from arXiv · showhide
We investigate neural techniques for end-to-end computational argumentation mining (AM). We frame AM both as a token-based dependency parsing and as a token-based sequence tagging problem, including a multi-task learning setup. Contrary to models that operate on the argument component level, we find that framing AM as dependency parsing leads to subpar performance results. In contrast, less complex (local) tagging models based on BiLSTMs perform robustly across classification scenarios, being able to catch long-range dependencies inherent to the AM problem. Moreover, we find that jointly learning 'natural' subtasks, in a multi-task learning setup, improves performance.
1 Introduction
The paper develops neural end-to-end approaches to computational argumentation mining, addressing the limitations of manually engineered pipeline systems. It compares dependency parsing, sequence tagging, multi-task learning, and a model combining sequential and tree information.
- 1 Introduction: Computational argumentation mining identifies argumentative units, classifies their types, finds relations, and classifies those relations.The subtasks include component segmentation, component classification, relation identification, and relation classification.
- 1 Introduction: Pipeline systems rely on manually adapted features and ILP constraints, while independently solving subtasks can cause error propagation.Datasets differ in how they annotate complex argument structures, making feature and constraint design challenging.
- 1 Introduction: The paper frames end-to-end AM as token-level dependency parsing without assuming argumentative boundaries or non-argumentative units are already identified.The dependency representation exploits the tree-like structure of argumentation while operating directly on tokens.
- 1 Introduction: The paper also frames AM as sequence tagging, encoding distances between linked components in tag labels to capture long-range dependencies.Related components can be several dozen tokens apart, which motivates neural sequence taggers.
- 1 Introduction: Multi-task learning adds natural AM subtasks as auxiliary tasks, and the study reports neural approaches that outperform the state-of-the-art joint ILP model.The contributions include robust neural tagging, ineffective token-based dependency parsing, and improved performance from auxiliary-task learning.
2 Related Work
Computational argumentation mining has been applied to legal decision making, document summarization, scientific-paper analysis, writing assistance, and essay scoring.
- 2 Related Work: AM supports applications in legal decision making, document summarization, and scientific-paper analysis.Its educational applications include writing assistance and essay scoring.
3 Data
The study uses a persuasive-essay corpus whose annotations represent major claims, claims, premises, and their support or attack relations. Its argument structures are nearly trees and are mostly contained within paragraphs.
- 3 Data: The PE dataset contains student essays responding to controversial topics such as whether competition or cooperation is better.The corpus is drawn from persuasive essays by students.
- 3 Data: The corpus contains 402 essays, including 80 reserved for testing.The supplied corpus-statistics passage identifies the dataset size and test split.
- 3 Data: Annotations distinguish major claims, claims, and premises, with 5,338 relations, more than 90% of which are supporting relations.The corpus includes 751 major claims, 1,506 claims, and 3,832 premises.
- 3 Data: Each premise links to exactly one claim or premise, making each essay’s argument structure almost a tree.Claims can connect to multiple major claims, which slightly violates a strict tree structure but can be losslessly re-linked.
- 3 Data: Argumentation is contained within paragraphs except possibly for claim-to-major-claim relations, making paragraph-level prediction easier than essay-level prediction.Paragraphs average 66 tokens, compared with 368 tokens for essays.
4 Models
The paper explores neural dependency parsing, sequence tagging, and multi-task learning for end-to-end AM. Sequence tagging encodes component and relation information in token labels, while alternative models combine sequential and tree representations.
- 4 Models: BiLSTMs assign labels using information from both left and right context, supporting sequence tagging with long-range dependencies.The model uses recurrent representations in which bidirectional LSTMs capture information on both sides of each token.
- 4 Models: The sequence-tagging label set encodes BIO boundaries, component type, relation distance, and relation stance for every token.Labels represent premises, claims, major claims, support, attack, and related distance information, with ⊥ for unfilled slots.
- 4 Models: In the PE data, about 30% of component relations have distance −1, around two-thirds lie in {−2, −1, 1}, and observed distances range from −11 to +10.The distribution motivates finite distance labels while demonstrating that long-range dependencies remain present.
- 4 Models: The multi-task setup trains auxiliary AM subtasks with distinct label sets alongside the main task.The framework uses auxiliary tasks within the same neural network.
- 4 Models: The dependency-parsing framing globally enforces a token-level tree, whereas tagging models are local and impose less structure on possible relations.LSTM-ER can model many-to-many relations, while STagT and dependency parsing restrict each token or component to one relation target.
- 4 Models: Sentence-level TreeLSTM information is ineffective for this dataset because roughly 92% of relations connect components in different sentences.The authors caution that enforcing a training-data constraint does not necessarily make a model more suitable.
5 Experiments
The experiments compare dependency parsing, sequence tagging, multi-task tagging, LSTM-ER, and ILP for end-to-end argumentation mining. Tagging models are generally more robust than parsers, while multi-task learning improves tagging performance.
- Dependency Parsing: The LSTM-Parser is over 100% better than Kiperwasser on exact component spans, but remains substantially weaker when evaluated on essays.Its essay-level performance also varies drastically across parametrizations.
- Dependency Parsing: Dependency parsers perform poorly on essay-level AM, with the LSTM-Parser reaching only 32.84%/47.44% C-F1 and 9.11%/14.45% R-F1.The reported values correspond to exact/approximate C-F1 and R-F1 levels, respectively.
- Sequence Tagging: STagBLCC outperforms the best parser by approximately 13% on C-F1 and 11% on R-F1 at paragraph level.The authors attribute this robustness to simpler local models that require less training data and better tolerate long sequences.
- Multi-Task Learning: MTL tagging improves performance by typically a few percentage points when component and relation subtasks are added as auxiliary tasks.The component-identification auxiliary task is consistently more helpful than the relation auxiliary task.
- LSTM-ER: LSTM-ER is 5% higher than STagBLCC on essay-level component C-F1 but 18% lower on relation R-F1, while substantially outperforming other systems at paragraph level.Its strong component performance is associated with decoupling component and relation tasks; STagBLCC achieves 95.34% versus LSTM-ER's 94.17% relation accuracy when both arguments are correct.
- Stability Analysis: Taggers have lower standard deviations than parsers, especially at essay level where parsers often nearly fail to learn.The stability comparison uses multiple runs with different random initializations and hyperparameters.
Error analysis
Exact argument-span segmentation is a systematic error source across systems. Neural taggers identify exact spans better than the ILP model and neural parsers, while humans also struggle with such boundaries.
- Exact argument-span detection is a systematic error source for all evaluated systems.The paper illustrates this with disagreements over discourse markers included in predicted versus gold premises.
- Neural taggers BLCC and BLC identify exact argument spans better than the ILP model and neural parsers.The comparison concerns exact identification, while the cited passage also reports substantially higher BIO macro-F1 for the neural taggers.
- Human annotators also have difficulty identifying exact entity boundaries.
6 Conclusion
The paper studies neural end-to-end argumentation mining through dependency parsing, sequence tagging, multi-task tagging, and models combining sequential and tree information. It reports strong neural results, especially for BiLSTM taggers, while identifying separate joint modeling of components and relations as preferable to naive coupling.
- The paper presents the first study of neural end-to-end computational argumentation mining.
- Neural models eliminate manual feature engineering and costly ILP constraint design while matching or surpassing a feature-based ILP formulation.
- BiLSTM taggers perform very well for component identification across the paper’s tagging frameworks.
- Component and relation identification should be modeled jointly but treated as separate tasks rather than naively coupled.
- An encoder-decoder framing is left for future work and is not evaluated in this study.
Supplementary Material
The supplementary material documents embeddings, hyperparameter optimization, evaluation tables, postprocessing corrections, and practical output constraints. Neural taggers and parsers are not guaranteed to satisfy all corpus-structure conditions, so simple correction rules were applied to violations.
- The experiments use pre-trained GloVe, syntactic, and structured skip n-gram embeddings.The listed GloVe dimensions are 50, 100, and 200.
- Dependency parsers and STagT taggers receive random hyperparameter optimization over 20 runs, whose systems are then averaged as an ensemble.
- The BiLSTM multi-task tagger uses 150 hidden units, 50-dimensional embeddings, 15 training iterations, and development-set model selection.
- The BiLSTM-CNN-CRF tagger samples hidden-layer size from {125, 150, 200, 250} and stops after five iterations without development improvement.
- The supplementary tables report F1 scores for BIO tagging and component segmentation plus classification.
- The models are not guaranteed to produce outputs satisfying corpus constraints, so violations receive simple postprocessing corrections.Examples include invalid BIO sequences, heterogeneous component links, out-of-range links, and parser links to non-argumentative units.
Error Analysis
The error analysis examines the strongest models, component-type detection, and relation errors. It finds that ILP substantially underperforms the LSTMs except for I-P component segmentation and classification.
- The analysis focuses on ILP, LSTM-ER, and STagBLCC as the three best models.
- Component segmentation and classification: O and I-P are particularly easy component classes, while claims and premises can be confused after misleading introductory phrases.
- Model comparison: ILP is substantially worse than both LSTMs in every case except I-P on component segmentation and classification.
- Relation errors: Relation errors often arise because one or both linked components do not match, while correctly matched components are mainly confused between Attack/Against and Support/For.Support/For relations comprise 94% and 82% of relations in the PE data, respectively.