PaperlayerView exact revision on arXiv

Source-linked AI summary

Who Argues What? Joint Argument-Entity Detection and Classification in Political Debates

Lucio La Cava, Stefano Francesco Monea, Sergio Greco

arXiv:2609.10192v1cs.CL

TL;DR

Political-debate argument mining lacks a paired entity layer, even though arguments refer to debate-relevant people, events, and organizations. The paper adds that layer through DNE-ElecDeb and introduces JAET, which inserts joint argument and entity tags into turns while preserving transcripts. JAET outperforms sequential AM-DNER pipelines, with reported relative F1 gains of +27.3% untyped and +41.9% typed.

  • Problem

    Political-debate AM resources typically annotate argumentative spans and roles without explicitly modeling the entities that make those arguments politically meaningful.

  • Method

    The paper creates DNE-ElecDeb and proposes JAET, a single-pass turn-level framework that jointly inserts argument and entity tags while preserving the original transcript.

  • Results

    +27.3% and +41.9% relative F1 gains are reported for joint AM+DNER tagging in untyped and typed settings over the strongest sequential pipeline.

  • Takeaways & Limitations

    The conclusion reports that JAET’s gains cannot be recovered by composing independent modules and remain effective under robustness and generalization assessments.

  • Takeaways & Limitations

    The entity inventory comes from U.S. presidential and vice-presidential debates, so roles, organizations, and discourse patterns may not generalize across political scenarios or cultures.

Abstract

from arXiv · show

Political debates are often analyzed through Argument Mining (AM) to investigate the key arguments that drive them. However, political arguments are rarely interpretable from argumentative spans alone, as claims and premises generally depend on the entities (e.g., people, events, locations, parties) they mention. Existing AM resources and methods typically annotate argumentative spans and roles, but do not provide a paired debate-entity layer for asking which Debate Named Entities (DNE), e.g., actors and events, are invoked within debates. In this work, we address these data and methodological gaps by (i) introducing DNE-ElecDeb, an entity-enriched version of the USElecDeb dataset that adds DNEs in both argumentative and non-argumentative spans and defines Debate Named Entity Recognition (DNER) as the task of detecting DNEs, and (ii) proposing Joint Argument and Entity Tagging (JAET), a generative framework that fine-tunes decoder-only LLMs to insert inline argument and entity tags into debate turns while preserving the original transcript. Under BIO-tagging evaluation, JAET improves relative F1 on the joint AM+DNER task by +27.3%, resp. +41.9%, under the untyped, resp. typed setting over the strongest sequential AM-DNER pipelines, demonstrating that such gains cannot be recovered by composing two independent modules. Notably, similar margins replicate on Persuasive Essays (+26.6%, resp. +52.7%), showing effective generalization to domains orthogonal to political debates. By unifying argumentative and entity-level representations within a single view, our contributions pave the way for richer political debates understanding.

1 Introduction

Political-debate argument mining needs an entity layer because argumentative spans depend on referenced actors, events, and organizations. The paper introduces DNE-ElecDeb and JAET to jointly represent and tag arguments and debate entities.

  • Political debates interleave speaker turns with references to opponents, parties, institutions, and organizations inside and outside argumentative spans.
  • Traditional argument-mining annotations identify argumentative spans and roles but typically omit the entities that make political arguments interpretable.
  • Independent AM and NER modules can lose mutual dependencies or produce incompatible boundaries because arguments and entities are intertwined.
  • DNE-ElecDeb extends USElecDeb with a paired debate-relevant entity layer covering argumentative and non-argumentative spans across 44 manually annotated debates.
  • JAET jointly predicts argument boundaries and labels with entity boundaries and types in a single turn-level pass.
  • +27.3% and +41.9% relative F1 gains occur for joint AM+DNER tagging in untyped and typed settings over the strongest sequential pipeline.

2 Related Work

Prior work develops argument-mining and named-entity-recognition methods, but these lines of research are rarely integrated for political debates. The paper positions its contribution as a joint argument-and-entity task with debate-specific data.

  • Argument mining includes segmentation, classification, relation identification, and relation classification subtasks, supported by increasingly neural and Transformer-based methods.
  • Political-debate resources include corpora annotated for argument components, fallacies, and social reactions, alongside studies applying AM to debate data.
  • LLM-based AM work reformulates argument and relation extraction generatively across domains but generally does not target political debates with entity extraction.
  • NER-oriented LLM methods address entity locating and typing, yet they do not model the argumentative layer required in AM scenarios.
  • Table 1 compares related work by debate use, dataset release, and whether it jointly predicts argument and entity boundaries and types.

3 Problem Definition

The problem is formulated as turn-level generation that preserves debate text while inserting argument and debate-entity annotations. JAET treats argument components and debate named entities as paired inline structures, including entities outside arguments.

  • A debate transcript is an ordered sequence of turns, each pairing speaker metadata with a token sequence.
  • The streaming setting restricts annotation to the current turn, excluding previous and future turns from the model input.
  • An argument component is a contiguous, non-overlapping token span labeled CLAIM or PREMISE and enclosed by corresponding tags.
  • A debate named entity is a contiguous, non-overlapping debate-relevant span assigned a type from a predefined inventory and tagged accordingly.
  • DNER detects and tags debate named entities, including entities located outside argumentative spans.
  • JAET maps each turn to an annotated version that preserves token order while inserting paired inline argument and entity tags.

4 JAET Mapping Learning

JAET learns a turn-level transformation that preserves debate text while inserting inline argument and entity tags. Decoder-only LLMs are fine-tuned on paired turns and gold tagged outputs using a normalized negative-log-likelihood objective.

  • JAET formulation: JAET formulates tagging as supervised text generation that preserves each input turn and adds inline argument and entity tags when appropriate.Training units contain no previous or future turns, so annotations rely only on the current turn’s content.
  • Training data: Training instances flatten debate turns into instruction-context-output examples, pairing speaker information and raw text with each turn’s gold tagged output.The resulting set follows the Alpaca format for instruction fine-tuning.
  • Learning objective: JAET optimizes decoder-only LLM parameters by minimizing the negative log-likelihood of tagged outputs across turn-level examples.Per-turn losses are aggregated over the flattened training set.
  • Learning objective: The objective penalizes changed source text, missing or hallucinated tags, and incorrect argument or entity labels.Normalization prevents longer turns from dominating optimization, while equal weighting gives each turn-level sample the same weight.
  • Model selection: The study uses publicly accessible 7–8B open-weight models to support scalable, low-resource, and privacy-preserving deployment scenarios.The evaluated models are Llama 3.1 8B Instruct, Mistral v0.3 7B Instruct, and Qwen 2.5 7B Instruct.

5 Data

DNE-ElecDeb enriches an existing U.S. political-debate corpus with debate-relevant entity annotations paired with argumentative labels. Its annotation workflow combines LLM-assisted tagging with human validation and uses turn-level train/test evaluation splits.

  • Dataset: DNE-ElecDeb enriches USElecDeb with debate-relevant entity annotations across transcripts from 44 U.S. presidential and vice-presidential debates held between 1960 and 2020.The corpus focuses on reciprocal discussion between Democratic and Republican candidates.
  • Entity annotations: The new entity layer covers debate-relevant mentions in both argumentative and contextual non-argumentative spans.The inventory includes entities central to interpreting political debates.
  • Annotation workflow: Entity types were consolidated through manual inspection and LLM-assisted exploration, followed by Gemini-based tagging with human validation.The inventory includes PERSON, ROLE, ORGANIZATION, PARTY, LOCATION, EVENT, DATE, and LAW.
  • Validation: κ = 0.919 was achieved for independent human agreement on token-level BIO label sequences, while combining expert annotations with Gemini consensus yielded κ = 0.900.Three domain experts independently annotated a stratified sample of raw transcripts while blind to LLM annotations.
  • Validation: κ = 0.862 was obtained after adding GPT-5.5 as an additional LLM annotator, and substituting human annotations as ground truth left every reported result unchanged.The authors interpret these findings as supporting the stability of the LLM-assisted annotation process.
  • Dataset split: The dataset uses an 80/20 turn-level train/test split designed to preserve an argument-rich test set for real-time debate analysis.This design evaluates turn-level performance rather than held-out-debate generalization.

6 Experimental Setup

The experiments compare JAET with prompt-based and task-specific alternatives for joint argument mining and debate entity recognition. Evaluation uses aligned macro-averaged BIO scores plus measures of transcript preservation and tag well-formedness.

  • Task definition: JAET outputs must preserve original turns, identify and classify argumentative spans, and recognize political entities and their roles.The task jointly evaluates argument and entity labels for the same token.
  • Compared methods: The study compares JAET with zero-shot and few-shot prompt-based baselines and task-specific decoder-only and encoder-only methods.Prompt baselines isolate instruction following, while competing methods provide architectural comparisons.
  • Evaluation: BIO evaluation reports macro-averaged precision, recall, and F1 under boundary-only and typed variants.Typed labels pair boundary markers with argument or entity classes, whereas boundary-only labels ignore span type.
  • Evaluation: Equal-length BIO alignment uses right-padding with O tags, penalizing generated outputs that insert, delete, or reorder transcript tokens.This enables direct token-wise comparison between predicted and gold sequences.
  • Generation quality: JAET evaluation also measures whether generated text preserves the source and whether tags are syntactically well formed.These checks address changed text, hallucinations, malformed tags, and invalid nesting in generative inline tagging.

7 Results

JAET variants outperform competing and prompt-based methods across argument mining, debate entity recognition, and joint tagging. Ablations indicate that joint modeling, rather than sequential composition, drives the joint-task gains without reducing argument-mining performance.

  • Mistral 7B achieves the strongest AM F1 of 0.745 and joint AM+DNER F1 of 0.644, while Llama 3.1 8B achieves the highest DNER F1 of 0.918.
  • +24% relative F1 improves AM over the strongest AM-only baseline, while +36.4% improves DNER over the strongest prompt-only baseline.For joint AM-DNER, JAET improves +87.2% over RooseBERT.
  • Few-shot prompting remains below JAET, despite improving over zero-shot for AM, DNER, and joint tagging.The best few-shot scores are 0.532 for AM, 0.673 for DNER, and 0.317 for joint tagging, versus JAET scores of 0.745, 0.918, and 0.644.
  • JAET preserves output quality, with text preservation at least 0.946 and tag well-formedness at least 0.998, while results vary below 10^-3 across three seeds.On correctly preserved turns, JAET retains a 0.19 AM-F1 and 0.26 joint-F1 advantage.
  • Ablations show that JAET preserves AM performance under untyped and typed evaluation, while joint modeling outperforms sequential AM-DNER and DNER-AM pipelines.These comparisons support the claim that single-pass prediction exploits argument-entity interdependency better than independent modules.
  • +27.3% relative F1 improves over the best AM→DNER pipeline in untyped joint tagging, rising to +41.9% in typed tagging.Against DNER→AM, the corresponding margins are +55.2% and +122%.

8 Robustness and Generalization

Robustness tests show that JAET's results are stable across annotation provenance and extend to unseen debates and a substantially different written-essay domain. These evaluations support generalization beyond the default turn-level political-debate split.

  • Replacing LLM-assisted entity annotations with annotations from three human experts changes joint untyped F1 only from 0.6436 to 0.6448.The joint scores slightly increase, indicating that performance is invariant to whether annotations are model- or human-produced.
  • A debate-level split with 35 training debates and 9 entirely held-out test debates evaluates generalization without potential turn-level leakage.The model becomes more conservative on unfamiliar spans, which the authors ascribe to genuine generalization rather than memorization.
  • On Persuasive Essays, JAET reaches untyped AM F1 = 0.8641 and joint F1 = 0.6844, improving over the strongest prompt baseline by +104% and +235.5%, respectively.The essays differ from political debates in being written, single-author, and edited rather than spoken, adversarial, and disfluent.

9 Conclusions

The paper contributes an entity-enriched political-debate dataset and JAET, a single-pass framework for joint argument and debate-entity tagging. Its evaluation reports gains over independent pipelines that persist under robustness and generalization assessments, while future work targets richer relations and cross-turn dependencies.

  • DNE-ElecDeb extends USElecDeb with debate entities, while JAET performs single-pass generative argument mining and debate named entity recognition.
  • Future work will extend the framework to argument and entity relations and richer cross-turn dependencies.

Limitations

The paper identifies language, cultural setting, turn-level context, and annotation provenance as boundaries on DNE-ElecDeb and JAET. It also leaves broader generalization and cross-talk resolution as open challenges.

  • DNE-ElecDeb is built from English-only U.S. presidential and vice-presidential debate transcripts, so discourse patterns may not transfer across languages or cultures.
  • The entity inventory derives from U.S. presidential and vice-presidential debates, limiting its demonstrated generalizability to other political scenarios and cultures.
  • JAET operates at the turn level because whole debates exceed target models’ context windows and larger windows degrade metrics and text preservation.
  • Turn-level processing limits detection of cross-talks and co-references, which the paper identifies as a direction for improvement.
  • The entity layer uses an LLM-assisted, human-validated workflow rather than exhaustive human annotation, which may introduce systematic tendencies.

Ethical Considerations

The paper describes implementation and annotation choices relevant to responsible interpretation, including automated entity annotation, sampled human validation, and the need to treat the released tools as analytical aids rather than decision-making systems.

  • The released dataset and framework may affect interpretations of candidates, parties, and public debates when used without human supervision.
  • The authors urge stakeholders to use the tools as analytical aids rather than decision-making tools and reject misuse or decisions based on them.
  • Entity annotations were produced through an LLM-assisted workflow and checked through human validation, with agreement assessed on a debate-stratified sample of 105 turns.
  • The annotation process includes documented ambiguity classes involving compound or nested entities, role references, event-location readings, and titled names.
  • The pipeline removes reformulations through transcript alignment and resolves segmentation differences through consensus and human validation; the paper reports no effect on final evaluation.

D Additional Results on Robustness

Additional experiments examine unseen-debate evaluation, transfer to Persuasive Essays, larger turn windows, and sequential-pipeline failure cases. They support JAET’s robustness across settings while showing that more context can reduce performance and structural fidelity.

  • Generalization to Unseen Debates: A strict debate-level split trains on 35 debates and tests on 9 entirely held-out debates, preventing test-debate turns from appearing during training.
  • Domain Transfer: On Persuasive Essays, JAET is evaluated under boundary-only and typed BIO settings, with an ablation showing its gain cannot be recovered by sequential AM–DNER pipelines.
  • Effect of the Turn-level Window: Both tagging quality and structural fidelity degrade monotonically as the input aggregates more consecutive turns.
  • Sequential pipelines fail because each stage cannot exploit the other layer’s signal, causing incompatible or incomplete argument-entity structures.
  • In representative cases, JAET recovers argument components and nested entity configurations that sequential pipelines omit, destroy, or mislabel.
Loading 2609.10192v1…