TTLab at StanceEval-2026: A Cloze-Style Prompting Approach for Arabic-Language Stance Detection (CLASP-Ar)

1 Introduction

• Social media’s rapid growth has accelerated misinformation and polarization, motivating automated analysis of social media content.The platforms also enable real-time global communication, particularly during emergencies.Source: §1 · p. 1
• Stance detection infers a user’s position toward a specific topic from textual posts, typically processing the target and text jointly.Prior research has focused primarily on English, while StanceEval 2026 extends the Arabic Mawqif benchmark.Source: §1 · p. 1
• CLASP-Ar combines masked language modeling, bidirectional encoder representations, and sentiment as an explicit feature, with constrained label prediction yielding an average gain of approximately 2 F1 points across stance labels.It is presented as a way to reduce the complexity of multitask-learning and ensemble approaches.Source: §1 · p. 1

2 Background

• Target-specific stance detection has been studied across languages, including English, Chinese, Turkish, Czech, Italian, and Arabic.StanceEval 2026 is an Arabic shared task.Source: §2 · p. 2

2.1 Dataset

• Mawqif-XT contains target topics and controversial texts annotated for stance, sentiment, and sarcasm, with Favor, Against, and None stance labels.Mawqif-XT is an extended version of Mawqif.Source: §2.1 · p. 2

2.2 Tracks

• Track 1 evaluates targets observed during training, whereas Track 2 evaluates unseen targets; ArabicStance-X provides 14,477 tweets across 17 topics for unseen-target support.ArabicStance-X is used for intermediate stance pre-training, while Mawqif-XT is used for development and evaluation.Source: §2.2 · p. 2, Table 1 · p. 2
• In the 2024 Arabic Stance Detection Shared Task, AlexUNLP-BH used multitask learning over sentiment and sarcasm, weighted cross-entropy with data augmentation, and ensembling; MGKM used fine-tuned large language models such as GPT-3.5-Turbo, and StanceCrafters combined two BERT encoders.Source: §2.2 · p. 2
• CLASP-Ar adds sentiment as an input feature in a single cloze-style prompt and uses a verbalizer-constrained MLM objective instead of multiple task heads or ensembles.The authors state that this design largely avoids multitask and ensemble complexity and is more suitable for transfer across application scenarios.Source: §2.2 · p. 2

3 System Overview

• Given a target, contextual sentiment label, and input text, CLASP-Ar constructs a prompt containing one [MASK] token for cloze-style masked language modeling.A BERT-based encoder processes the sequence.Source: §3 · p. 2
• The MLM head is restricted to the stance labels Favor, Against, and None, represented by the single-character verbalizer tokens ‘F’, ‘A’, and ‘N’.The tokens correspond to the initial letters of the class names.Source: §3 · p. 3
• CLASP-Ar computes stance-label probabilities by applying a softmax over the constrained vocabulary Vc = {‘F’, ‘A’, ‘N’} to the [MASK] representation.The model is trained end-to-end with cross-entropy against the ground-truth stance label.Source: §3 · p. 3

4 Experimental Setup

• To address scarce None labels, the setup uses class-weighted cross-entropy and adds 1,815 None instances by pairing texts with randomly selected, unrelated targets.The setup describes the target pairings as mismatched and unrelated.Source: §4 · p. 3
• CLASP-Ar uses the aubmindlab/bert-base-arabertv02-twitter encoder and two training phases: intermediate pre-training on ArabicStance-X followed by fine-tuning on augmented Mawqif-XT.SARF generates sentiment values for ArabicStance-X, which lacks sentiment annotations; fine-tuning runs for up to 20 epochs with early stopping.Source: §4 · p. 3
• Evaluation measures accuracy and class-specific F1, with Favg2 for the two polar classes and Favg3 for all three classes.Table 2 reports final evaluation performance separately for Track 1 (seen targets) and Track 2 (unseen targets).Source: §4 · p. 3, Table 2 · p. 3

5 Results and Discussion

• CLASP-Ar demonstrates superior performance on Track 2 (unseen targets) in the official evaluation.The supplied passage does not include the associated metric values.Source: §5 · p. 3

5.1 Impact of Class Imbalance Mitigation Strategies

• Class weighting improved None performance by approximately 7 F1 points and Favg3 by approximately 2 points over baseline.Augmented None instances mostly improved Against but consistently harmed None; both strategies boosted performance over baseline and base configuration.Source: §5.1 · p. 4, Table 3 · p. 4

5.2 Impact of Auxilliary Sentiment and Sarcasm Information

• Sentiment alone raised Favg2 by 1.38 points over baseline (86.28% vs. 84.90%) and achieved the highest overall accuracy (84.38%).Combining sentiment and sarcasm yielded the highest Favg3 (72.50%).Source: §5.2 · p. 4, Table 4 · p. 4

5.3 Sentiment Source

• SARF raised development Favg2 from 80.96% to 81.50% versus the off-the-shelf sentiment model, so the study adopted SARF.SARF combines surface, stemmed, and rooted morphological representations from a shared MARBERTv2 encoder.Source: §5.3 · p. 4

• CLASP-Ar is a cloze-style masked language modeling framework tailored for stance detection.Source: §6 · p. 4
• Incorporating auxiliary signals, particularly sentiment, improved Favg2 by up to 1.38 points.Source: §6 · p. 4
• Class weighting boosted performance on the underrepresented None class by up to 9 F1 points.Source: §6 · p. 4
• On the official StanceEval-2026 benchmark, CLASP-Ar scored 71.36% Favg2 on Track 1 and 74.14% on Track 2.Track 1 evaluates seen targets; Track 2 evaluates unseen targets.Source: §6 · p. 4

• None-class performance is low and volatile, while unrelated-target augmentation may miss genuine non-committal stances and degrade Against precision boundaries.In some configurations, the standard deviation for Fnone exceeds 10 points.Source: Limitations · p. 5

A Leveraging ArabicStance-X for Stance Detection

• The transfer-learning comparison evaluates sequential ArabicStance-X pretraining and joint training on the union of ArabicStance-X and Mawqif-XT.Table 5 reports development Favg2 and includes the original shared-task large-model baseline.Source: A Leveraging ArabicStance-X for Stance… · p. 6, Table 5 · p. 6

B Class Imbalance Mitigation

• Class-imbalance experiments compare a base configuration, extended None instances, and class weights across three seeds on the development set.The study also reports the strategies’ impact across targets in Table 7.Source: B Class Imbalance Mitigation · p. 6

B.1 Data Augmentation vs. Loss Weighting

• 52.83% Fnone with pre-training was the highest none-class score under Class Weights, while Extended None reached 38.55% Fnone without pre-training and showed σ = 10.81.The authors associate the augmented instances with feature noise that complicates convergence in the minority class space.Source: §B.1 · p. 6, Table 6 · p. 7
• 86.33% Favg2 without pre-training was the highest polar-class score, achieved by Extended None, which the authors say isolates favor and against stances.The authors attribute this to saturating the none distribution during training, creating a stricter decision boundary for the polar classes.Source: §B.1 · p. 6
• 84.23% Favg2 and 69.44% Favg3 were achieved by pre-trained Extended None on Covid Vaccine, whose polar classes were balanced at approximately 43% each.The target-level comparison reports mean and standard deviation over five runs.Source: §B.1 · p. 7, Table 7 · p. 7
• For targets with highly skewed distributions and a very small none class, Class Weights outperformed augmentation; the authors favor it for global macro-performance, while augmentation suits balanced polar classes.The examples given are Women Empowerment, with approximately 5% none, and Digital Transformation, with approximately 10% none.Source: §B.1 · pp. 6–7

C Auxiliary Feature Ablations

• Three-seed development-set experiments assessed whether sentiment and sarcasm features help stance detection.The seeds were 42, 123, and 456.Source: C Auxiliary Feature Ablations · p. 7
• Table 8 reports label-specific F1 scores, Favg2, Favg3, and accuracy across the multi-seed development-set experiments.Source: Table 8 · p. 8