Source-linked AI summary
PAST-TIDE: Prototype-Anchored Statement Tuning with Topic-Invariant Normalization for Stance Detection
Md. Shakhoyat Rahman Shujon, MD Jahid Hasan Jim, Md. Milon Islam, Md Rezwanul Haque, Fakhri Karray
TL;DR
Multilingual stance detection remains underexplored when labels are limited, especially outside English. PAST-TIDE uses statement tuning with prototype-based contrastive learning and topic-conditional normalization, achieving official macro-F1 scores of 0.75 and 0.74 on the two subtasks.
Problem
Multilingual stance detection remains underexplored, particularly for prompt-based methods with limited labels in non-English languages.
Method
PAST-TIDE accesses the pre-trained MLM head through statement tuning, adding prototype-based contrastive learning and topic-conditional normalization.
Results
PAST-TIDE achieved official leaderboard macro-F1 scores of 0.75 for Subtask A and 0.74 for Subtask B.
Takeaways & Limitations
At this small data scale, the simpler proposed models outperformed larger models with added parameters.
Takeaways & Limitations
The authors plan to automate verbalizer selection and evaluate PAST-TIDE on benchmarks with finer-grained labels.
Abstract
from arXiv · showhide
We introduce PAST-TIDE, our stance detection system addressing both subtasks of the StanceNakba Shared Task at NakbaNLP@LREC-COLING 2026. The main idea is statement tuning. We redefine stance as cloze-style masked language modeling (MLM), letting a verbalizer map label words to stance categories through the pre-trained MLM head rather than appending a randomly initialized classification head. We complement this with prototypical contrastive learning, which uses learnable class prototypes for batch-size independent contrastive training, and topic-conditional layer normalization for cross-topic Arabic stance detection. PAST-TIDE achieves macro-F1 scores of 0.75 for Subtask A and 0.74 for Subtask B on the official leaderboard, indicating that minimal architectural additions to a pre-trained model can remain competitive in low-resource settings.
1. Introduction
PAST-TIDE addresses multilingual stance detection by accessing the pre-trained MLM head through statement tuning instead of relying on a randomly initialized [CLS] classifier head. It combines this approach with prototypical contrastive learning and achieves macro-F1 scores of 0.75 and 0.74 on the official Subtask A and B leaderboards.
- Motivation: Multilingual stance detection remains challenging, and the pre-training/fine-tuning gap of standard [CLS] classifier heads degrades performance at this scale.The field has progressed from handcrafted features to pre-trained fine-tuning and prompt-based reformulations.
- Approach: PAST-TIDE accesses classification capacity in the pre-trained MLM head rather than replacing it with a randomly initialized [CLS] head.This design is the system’s central idea.
- Approach: Statement tuning reformulates stance detection as cloze-style MLM and uses a multi-token verbalizer averaging predictions across register-diverse label words.The method bypasses the randomly initialized [CLS] head.
- Approach: PAST-TIDE combines statement tuning with prototypical contrastive learning, which replaces in-batch contrastive training with learnable class prototypes.The supplied passage identifies prototypical contrastive learning as a major contribution.
- Results: 0.75 macro-F1 placed PAST-TIDE 11th in Subtask A, while 0.74 macro-F1 placed it 7th in Subtask B on the official leaderboard.During testing, the model scored macro-F1 of 0.79 on both subtasks.
2. Related Work
Prior stance-detection research has focused mainly on English, while Arabic resources remain limited; prompt-based cloze-style MLM classification has shown consistent few-shot gains.
- English stance detection: Stance detection research has primarily focused on English, spanning SemEval-2016, cross-target generalization, and surveys through 2020.The cited work includes Mohammad et al. (2016), Allaway and McKeown (2020), and Küçük and Can (2020).
- Arabic stance detection: Arabic stance work remains limited, with MARASTA identified as one of the few multi-dialectal resources.MARASTA is attributed to Charfi et al. (2024).
- Prompt-based classification: Prompt-based classification through cloze-style MLM has shown consistent few-shot gains.The passage cites Schick and Schütze (2021) and Gao et al. (2021).
3. System Description
PAST-TIDE reframes stance detection as cloze-style masked language modeling, reusing mDeBERTa’s pre-trained MLM head and verbalizer instead of adding a randomly initialized classifier. PCL, topic-conditional normalization, and R-Drop address small-batch optimization, cross-topic variation, and decision-boundary regularization.
- Architecture: PAST-TIDE uses mDeBERTa-v3-base with 280M parameters and hidden dimension 768, sharing a single checkpoint across both subtasks.T-CLN is activated only for Subtask B.
- Statement tuning: Stance detection becomes a cloze-style MLM task: the pre-trained MLM head processes the [MASK] state, and a verbalizer aggregates relevant vocabulary-token log-probabilities into class probabilities.This replaces a randomly initialized [CLS]-based classification head and requires zero new parameters.
- Prototypical contrastive learning: PCL uses K=3 learnable class prototypes to pull each [MASK] embedding toward its ground-truth prototype, keeping the contrastive denominator fixed even at batch size 1.Unlike SupCon, PCL avoids dependence on in-batch negatives; SupCon had ∼5.3 expected negatives at batch size 8 and fold-level F1 variation of up to ±8 points.
- Topic-conditional normalization: T-CLN replaces standard layer-normalization parameters with topic-conditioned parameters for Subtask B, using identity-initialized networks that learn topic-specific shifts after the final transformer layer.Without identity initialization, the normalization parameters diverged early and destabilized the MLM head.
- Regularization and training: R-Drop regularizes the decision boundary through two dropout-masked forward passes and their symmetric KL divergence, alongside focal loss and PCL in the total training objective.The loss is L = Lfocal + 0.1 · LPCL + 1.0 · LR-Drop.
4. Experimental Setup
The experiments use the StanceNakba 2026 dataset for two stance subtasks, with training augmented by back-translation and evaluated through stratified cross-validation. Arabic preprocessing and memory-conscious optimization complete the setup.
- Dataset: The StanceNakba 2026 dataset contains 1,401 English samples for Subtask A and 1,205 Arabic samples for Subtask B, split 70/15/15.Subtask A addresses actor-level stance toward the Israeli-Palestinian conflict with Pro-Palestine, Pro-Israel, and Neutral classes; its training set has 980 samples.
- Augmentation and Ensembling: Training data is doubled through MarianMT back-translation via German for Subtask A and English for Subtask B.The German pivot is intended to encourage paraphrasing because of divergent word order, while evaluation uses 5-fold stratified cross-validation with early stopping at patience 3 and probability averaging across folds.
- Preprocessing: Arabic inputs are normalized before tokenization by unifying Alef and Ya forms, removing diacritics, and deleting Tatweel characters.
- Implementation: Experiments run on Dual NVIDIA T4 GPUs with an effective batch size of 32, a 256-token maximum sequence length, FP16, and gradient checkpointing.Optimization uses AdamW with a 10% linear warmup followed by cosine decay.
5. Results and Analysis
PAST-TIDE’s testing scores reached 0.79 macro-F1 on both subtasks, while official leaderboard scores were lower at 0.75 and 0.74. Analysis attributes performance to statement tuning, topic-conditional normalization, and minimal parameter overhead, while identifying neutral-post confusion as a key error pattern.
- Overall Results: 0.79 macro-F1 was achieved on both subtasks during testing, while official leaderboard scores settled at 0.75 / 0.74.Testing used a subset of development data; final evaluation used the held-out test partition.
- Error Analysis: In Subtask A, 11 of 14 Neutral errors were classified as Pro-Palestine, while the two main classes were confused almost equally, 8 vs. 7.The passage attributes Neutral confusion to overlap between ceasefire and civilian-death language and Pro-Palestinian posts.
- Cross-Topic Transfer: Removing T-CLN had no effect on Subtask A but reduced Subtask B by 6% during testing, indicating topic-specific representation differences across the Arabic topics.The passage describes conditional normalization as bridging those topic representations.
- Parameter Overhead: Subtask A added 2,304 parameters and Subtask B added ∼1.28M for T-CLN, while the MLM head and verbalizer added zero new weights.The passage states that classification capacity comes from weights pretrained on hundreds of gigabytes of multilingual text.
6. Conclusion
PAST-TIDE achieved competitive official leaderboard performance by reusing the pre-trained MLM head, while its modules support label-flexible, small-batch, and multi-domain settings. Future work will automate verbalizer selection and evaluate benchmarks with finer-grained labels.
- Conclusion: 0.75 and 0.74 macro-F1 were obtained on the official Subtask A and B leaderboards by reusing rather than replacing the pre-trained MLM head.Macro-F1 reached 0.79 on both subtasks during testing.
- Conclusion: The simpler proposed models outperformed larger models with added parameters.
- Conclusion: Statement tuning supports arbitrary label sets with real vocabulary words, PCL supports settings with small batches, and T-CLN supports multi-domain problems.These correspond to the three proposed architectural modules.
- Conclusion: Future work will automate verbalizer selection through gradient-based token search and test benchmarks with finer-grained labels.
Bibliographical References · A. Hyperparameters
The supplied passage identifies Table 4 as presenting the hyperparameter configurations for PAST-TIDE.
- A. Hyperparameters: Table 4 presents the hyperparameter configurations for PAST-TIDE.
B. Limitations
The study’s component improvements are not isolated by controlled ablation, and its intuitive verbalizer selection remains unautomated. Generalization is also uncertain beyond the two geopolitical topics and their label structure.
- Experimental limitations: Compute limits prevented a controlled single-architecture ablation, so component improvements may be confounded by changes between runs.The experiments used dual T4 GPUs for 9 hours on Kaggle; a proper ablation study would strengthen the results.
- Experimental limitations: Verbalizer tokens were selected using intuition about stance-relevant vocabulary rather than an automated search.The passage identifies automated methods such as gradient-based token selection as a possible alternative.
- Generalization limits: The data covers only two specific geopolitical topics, leaving transfer to domains with different label semantics or class counts untested.The architecture’s performance across domains with different numbers of classes remains unknown.
C. Ethics Statement
The authors clarify that PAST-TIDE’s stance predictions reflect surface-level linguistic cues, not the authors’ views, and that all data use followed shared-task guidelines.
- C. Ethics Statement: PAST-TIDE infers stance from surface-level linguistic cues, without implying the authors’ own views on the topics.All data were used strictly according to the shared task guidelines.