Source-linked AI summary
TTLab at Daleel 2026: STAR-Ar, Sequence Tagging for Argument Recognition in Arabic
Bhuvanesh Verma, Ali Abusaleh, Alexander Mehler
TL;DR
Arabic argument mining remains under-resourced, and Daleel 2026 asks systems to identify and classify ADUs in Arabic texts. STAR-Ar uses BERT-BiLSTM-CRF sequence tagging for this task, achieving a test F1 score of 73.74 and ranking third among 14 teams. The authors note constraints from the input-length limit and generalization across Arabic text types and dialects.
Problem
Method
STAR-Ar jointly detects and classifies ADUs as a token-level sequence-labeling task using a BERT-BiLSTM-CRF architecture.
Source: §1 · p. 1Results
73.74 F1 on the final test set in the both-domain setting ranked STAR-Ar third among 14 participating teams.
Source: §6 · p. 4Takeaways & Limitations
The authors conclude that sequence labeling remains a competitive, highly efficient paradigm for joint span detection and classification in argument mining.
Source: §6 · p. 4A 512-subword input limit constrains applicability to longer documents, and generalization to other Arabic text types and dialectal variations remains a limitation.
Source: Limitations · p. 5
Abstract
from arXiv · showhide
Argument Mining (AM) is a critical NLP task that remains significantly under-resourced in Arabic. This paper presents $\testtt{STAR-Ar}$, a BERT-BiLSTM-CRF architecture for argument discourse detection and classification, as our system for Daleel 2026, the inaugural Arabic argument mining shared task. The task requires the identification and classification of argumentative discourse units (ADUs) in debate and editorial texts.We jointly model these two objectives as a token-level sequence labeling task using a BERT-BiLSTM-CRF architecture that combines contextual transformer embeddings with structural transition constraints to support accurate span detection. $\testtt{STAR-Ar}$ achieves an F1-score of 72.69 on validation and 73.7 on test data. Our domain-specific analysis shows that models trained exclusively on editorials underperform those trained on debates, a disparity we primarily attribute to the smaller size of the editorial dataset. The code for $\testtt{STAR-Ar}$ is available at ${\href{https://github.com/ENTAILab/daleel_2026_Arabic-Argumentative-Discourse-Mining}{\faGithub~TTLab at Daleel 2026}}$
1 Introduction
Argument mining extracts argumentative structures, but Arabic remains under-resourced. STAR-Ar addresses Task 2 of Daleel 2026 with a sequence-tagging system for detecting and classifying argumentative discourse units.
1 Introduction
- Argument mining extracts argumentative structures and reasoning; claims and premises are primary targets in many modern systems.The Toulmin Model identifies claims, premises, and warrants as primary argument components.Source: §1 · p. 1
- Argumentative discourse units are minimal atomic argument spans, and extraction requires both precise boundary detection and classification.ADUs may range from a short dependent clause to several consecutive sentences.Source: §1 · p. 1
- Daleel 2026 introduces an Arabic benchmark with paragraph-level classification and token-level span detection tasks; STAR-Ar focuses on Task 2 and ranks third among 14 teams.The system is presented as a stable, mathematically constrained baseline with cross-domain performance on debates and editorials.Source: §1 · p. 1
2 Background
Daleel 2026 addresses Arabic argument mining through two tasks: paragraph-level ADU classification and token-level ADU span detection. The paper focuses on the latter, building on sequence-tagging work for argument mining.
2 Background
- Arabic remains under-resourced for argument mining, and Daleel 2026 introduces two tasks for identifying and classifying Arabic argumentative discourse units.The paper describes Task 1 as paragraph-level multi-label classification and Task 2 as token-level span detection.Source: §2 · p. 2
2.1 Task 1: ADU Paragraph Classification
- Task 1 predicts one or more ADU classes per paragraph, with Assumption present in 75.5% of training paragraphs and Statistics in 4.6%.Paragraphs have an average of 2.0 labels.Source: §2.1 · p. 2
2.2 Task 2: ADU Span Detection
- Task 2 tags tokens to identify and classify exact ADU character spans, with 2,975 training spans and dense labeled coverage in many paragraphs.84.6% of training paragraphs have 80%–100% of their text volume within labeled spans; the dataset is divided into train, development, and test sets.Source: §2.2 · p. 2
- For extracting and classifying ADUs in Arabic texts for the Daleel 2026 shared task, the paper uses the framework established by Binder et al. (2022) as its foundational methodological basis.Table 1 reports paragraph-level label coverage in the Daleel 2026 dataset.Source: §2.2 · p. 3, Table 1 · p. 3
3 System Overview
STAR-Ar formulates ADU detection and classification as BIO sequence tagging, combining an Arabic-pretrained encoder, a BiLSTM, and a CRF. Its cost-sensitive composite loss pairs the CRF sequence objective with auxiliary token-level cross-entropy.
3 System Overview
- STAR-Ar treats ADU detection and classification as BIO sequence tagging, using a pre-trained transformer, recurrent layer, and structured prediction head.The formulation is intended to address the dataset’s imbalance and structured nature.Source: §3 · p. 3
3.1 Model Architecture
- The architecture passes Arabic BERT representations through a BiLSTM and linear BIO classifier, then uses a CRF with prohibitive scores for invalid transitions.Penalized paths include initial I-tags, O→I transitions, and cross-label continuations; the Viterbi decoder therefore outputs structurally coherent sequences.Source: §3.1 · p. 3
3.2 Optimization and Loss Formulation
- The cost-sensitive loss combines CRF sequence negative log-likelihood with auxiliary token-level cross-entropy, weighted by inverse-square-root class weights.The O-tag weight is capped at 0.3, and λaux controls the auxiliary loss weight.Source: §3.2 · p. 3
4 Experimental Setup
STAR-Ar uses stratified cross-validation, selects MARBERTv2 as its encoder, and trains the final system on train and development data. Test predictions average fold-level CRF emission probabilities before a single Viterbi decode.
4 Experimental Setup
- Five-fold stratified cross-validation distributes documents according to their least frequent label, while document-level weighted sampling oversamples paragraphs with minority ADU classes.Source: §4 · p. 3
- MARBERTv2 outperformed both AraBERT variants across multi-seed experiments and was selected as the final encoder.The setup also uses a two-layer BiLSTM with 256 hidden units per direction, AdamW, and early stopping.Source: §4 · p. 3
- Table 2 reports F1-score (%) on development and test sets, organized by training domain and evaluation split and domain.Source: Table 2 · p. 4
- The final system trains on both train and development sets, averages raw CRF emission probabilities across folds, then performs one Viterbi decoding step.This combines fold predictions at the emission level rather than by majority voting on decoded sequences.Source: §4 · p. 4
5 Results and Discussion
STAR-Ar performs better on test than development data, while domain results favor debate-trained models and show that debate data can improve editorial performance. Span-level analysis also reveals a strong bias toward Assumption predictions.
5 Results and Discussion
- Debate-trained models perform better on debates, while editorial-only models benefit more from debate data, likely because the editorial subset is smaller and more imbalanced.Training on debate alone yields performance close to training on both domains.Source: §5 · p. 4
- Figure 2 reports mean character-wise overlap by gold class; its diagonal represents span-level recall, while NONE captures false negatives.The confusion matrix visualizes the span-level boundary analysis.Source: Figure 2 · p. 4, §5 · p. 4
- Assumption achieves 88.49% recall but absorbs 74.22% of gold Common ground characters, leaving Common ground recall at 12.26%.Statistics, despite being rarest, has 64.77% recall; its explicit quantitative cues may help it resist absorption.Source: §5 · p. 4
6 Conclusion
STAR-Ar is a BERT-BiLSTM-CRF system for Arabic ADU extraction and classification. It ranks third on the shared-task test set, while the analysis notes class-wise and cross-domain challenges.
6 Conclusion
- 73.74 F1 on the both-domain final test set ranked STAR-Ar 3rd among 14 teams.Source: §6 · p. 4
- The authors characterize sequence labeling as a competitive, highly efficient paradigm for joint span detection and classification in argument mining.Source: §6 · p. 4
- Class re-balancing yields non-uniform gains, label frequency alone does not predict class difficulty, and cross-domain evaluation shows high sensitivity to domain shifts.The authors call for larger, more structurally diverse annotated corpora to advance Arabic argument mining.Source: §6 · pp. 4–5
Limitations
The 512-subword limit truncates some inputs, and the authors identify longer documents and broader Arabic text types and dialects as applicability boundaries.
Limitations
- The 512-subword limit truncates long inputs, constraining applicability to longer documents.Source: Limitations · p. 5
- Truncation affects 1.14% of training, 0.46% of development, and 0.47% of test data, which the authors describe as negligible portions.Source: Limitations · p. 5
- The authors also identify generalization across other Arabic text types and dialectical variations as a limitation.Source: Limitations · p. 5
A Encoder Comparison
The encoder comparison evaluates pretrained Arabic text encoders using multi-seed, stratified cross-validation. Table 3 describes an overall-performance ablation on the combined dataset.
A Encoder Comparison
- Three pretrained Arabic text encoders were compared using the same five-fold stratified cross-validation across three seeds.Source: A Encoder Comparison · p. 5
- The reported comparison uses the training set and applies stratification across folds; the supplied passages do not state which encoder performed best.Source: A Encoder Comparison · p. 5, Table 3 · p. 5
- Table 3 presents an ablation of pretrained text encoders’ impact on overall performance using models trained on the combined dataset.Source: Table 3 · p. 5
B Paragraph Level Error Analysis
The development-set analysis finds that errors on minority classes are more often omissions than confusion with competing labels. Statistics is highly precise despite its rarity, while Common ground remains difficult to recover.
B Paragraph Level Error Analysis
- Minority-class errors are predominantly omissions; Anecdote and Common ground are most often classified as None rather than competing labels.The analysis uses the development-set confusion matrix.Source: B Paragraph Level Error Analysis · p. 5, Figure 3 · p. 6
- 100% precision: Statistics recovers 6 of 9 instances without false positives, with explicit quantitative features providing a separable signal exploited by cost-sensitive weighting.Source: B Paragraph Level Error Analysis · p. 5
- Common ground has the lowest recall, recovering 2 of 12 instances; the pattern persists in single-domain configurations, while multi-domain training mitigates cross-domain sparsity through complementary label distributions.Source: B Paragraph Level Error Analysis · p. 6