Source-linked AI summary
A Survey on Stance Detection for Mis- and Disinformation Identification
Momchil Hardalov, Arnav Arora, Preslav Nakov, Isabelle Augenstein
TL;DR
False information has made effective online detection increasingly important, and stance detection is a core NLP task in that effort. This survey reviews how stance is formulated, represented in datasets, and applied to mis- and disinformation detection, concluding with lessons and future directions. It highlights a research landscape shaped by small, imbalanced datasets, varied task definitions, and challenges in modelling context and language.
Problem
No existing overview examined how different stance-detection formulations contribute to detecting false content, despite stance’s role in fact-checking and related systems.
Method
The survey synthesizes task formulations, datasets, methods, lessons, and future research on stance detection for mis- and disinformation identification.
Results
The reviewed research spans direct fact-checking and pipeline-based uses, with performance and comparability shaped by differing datasets, task formulations, and evaluation settings.
Takeaways & Limitations
Future progress requires addressing limited and imbalanced datasets, varying label inventories, multilingual coverage, and richer context modelling.
Takeaways & Limitations
Most stance datasets contain at most a few thousand examples and often have skewed class distributions with few minority-class instances.
Abstract
from arXiv · showhide
Understanding attitudes expressed in texts, also known as stance detection, plays an important role in systems for detecting false information online, be it misinformation (unintentionally false) or disinformation (intentionally false information). Stance detection has been framed in different ways, including (a) as a component of fact-checking, rumour detection, and detecting previously fact-checked claims, or (b) as a task in its own right. While there have been prior efforts to contrast stance detection with other related tasks such as argumentation mining and sentiment analysis, there is no existing survey on examining the relationship between stance detection and mis- and disinformation detection. Here, we aim to bridge this gap by reviewing and analysing existing work in this area, with mis- and disinformation in focus, and discussing lessons learnt and future challenges.
1 Introduction
The survey examines how stance detection supports identifying mis- and disinformation online, addressing the lack of an overview focused on their relationship. It reviews existing task settings, datasets, and methods while situating stance detection among related applications.
- Motivation: Social media growth has enabled dubious content to spread during controversial events and the COVID-19 infodemic.These developments motivate the need for effective online mis- and disinformation detection.
- Scope: The survey studies stance detection as the NLP task underlying fact-checking, fake-news, rumour, and hoax detection.It distinguishes misinformation from disinformation, which additionally involves an intention to harm.
- Related applications: Stance detection can support ideological-debate analysis, issue framing, media-leaning identification, and fact-checking across varied text types.Prior work has applied it to political debates, fact-checking, and product-related texts.
- Research gap: Existing surveys cover rumour verification, social-media stance, stance detection generally, and fact-checking, but not their relationship to false-content detection.This survey addresses that missing overview.
- Survey contribution: The survey covers task formulations, datasets, and methods for stance detection in mis- and disinformation research, then derives lessons and future trends.Its framing includes stance as fact-checking itself or as a component within a veracity-assessment pipeline.
2 What is Stance?
Stance detection is presented as the classification of an author’s position toward a target, while stance-taking also reflects social and cultural influences. The survey distinguishes this task from several related NLP tasks.
- Stance concept: Stance expresses a speaker’s standpoint and judgement toward a proposition, and stance-taking is shaped by social interaction and external factors.The cited account includes evaluating objects, positioning subjects, and aligning with others.
- Operational definition: Given text and a target, stance detection classifies the author’s position into Favor, Against, or Neither, sometimes adding Neutral.The target may be explicit or implicit, although the supplied passage ends before describing all target forms.
- Related tasks: Stance detection differs from biased-language, emotion, perspective, and sarcasm detection, which target different properties of text or authors.The supplied passage introduces these distinctions but does not provide the complete list of related tasks.
3 Stance and Factuality
The survey organizes stance-based factuality research around two formulations: stance as fact-checking and stance as a component of a broader veracity pipeline. It compares evidence settings, datasets, and rumour-oriented applications.
- Formulations: Stance detection can perform fact-checking directly or operate as one component of a fact-checking pipeline.The pipeline may also model users or profile media outlets.
- Dataset organization: Datasets are characterized by their source, target, evidence type, and task, with evidence ranging from single documents to multiple documents or conversational threads.The survey also presents a four-level taxonomy covering sources, inputs, categorisation, and textual object types.
- Fact-checking as stance detection: Standalone fact-checking tasks infer veracity-related labels from the stance of evidence toward a target claim.FNC-1 used positive, negative, discuss, and unrelated labels for article-body and headline relatedness, while FEVER used SUPPORTED, REFUTED, and NOT ENOUGH INFO for claims and Wikipedia evidence.
- Evidence complexity: FEVER supports multi-hop reasoning over potentially composite claims, whereas Snopes-based evidence stance does not require multi-hop reasoning.FEVER’s NOT ENOUGH INFO category covers claims that available Wikipedia information cannot support or refute.
- Task boundaries: FEVER differs from typical stance detection because it evaluates evidence supporting or refuting claims rather than an author’s stance toward them.The survey connects FEVER to argument reasoning and Natural Language Inference while still relating it to stance detection through its resemblance to FNC.
- Rumour detection: Rumour detection uses labels such as Support, Deny, Query, and Comment to interpret reactions from users, media, or other sources toward uncertain claims.RumourEval provided stance and veracity annotations and showed the importance of modelling conversational context rather than relying on a single post.
- Rumour research development: Early rumour work demonstrated feasibility but was limited by five rumours and a focus on individual posts, while later work examined threads, confirmation status, and rumour veracity.The later studies included nine newsworthy events and tracked stance before and after rumours were confirmed or denied.
- News-based evidence: The Emergent project used journalists’ stance and veracity annotations, but required news articles to be summarized into single sentences.This summarization requirement contrasts with FNC-1’s use of entire news articles.
4 Approaches
The surveyed approaches use stance detection either directly for fact-checking or within broader mis- and disinformation pipelines. Results show benefits from pretrained and graph-based models, but dataset imbalance, thread structure, retrieval, and inconsistent task definitions remain important constraints.
- Fact-Checking as Stance Detection: Fact-checking approaches model veracity through stance labels, using evidence documents, retrieved snippets, or claim–article relations.FEVER-style systems add evidence retrieval, while other tasks predict article stance toward claims directly.
- Fact-Checking as Stance Detection: Pretrained Transformers substantially improve imbalanced stance classes, with RoBERTa raising Disagree F1 from 18 to 58 and Agree F1 from 50 to 70.Language-specific and multilingual pretraining also achieves strong results across Arabic and cross-lingual datasets.
- Fact-Checking as Stance Detection: Graph-based fact-checking models improve FEVER performance to 70, while retrieval quality remains consequential and gold evidence adds 1.4 points.These systems support multi-hop reasoning and propagate information across evidence graphs.
- Threaded Stance: Threaded stance models exploit conversational structure, but rumour detection suffers from class imbalance and declining performance at greater thread depth.The denying class is especially difficult, whereas incorporating stance labels through multi-task learning can substantially boost veracity detection.
- Threaded Stance: Temporal analysis finds that source posts usually support rumours, denials emerge later, and querying posts become proportionally more common over time.These patterns motivate models that account for both thread structure and post timing.
- Multi-Dataset Learning: State-of-the-art results are difficult to compare because datasets differ in task formulation, labels, class balance, topics, size, and evaluation measures.Combining datasets also remains challenging because annotation schemes and label inventories vary.
5 Lessons Learned and Future Trends
The survey identifies dataset, label, explainability, integration, multilinguality, context, multimodality, and truth-granularity challenges, alongside directions for addressing them.
- Dataset Size: Existing stance datasets are usually small and often have skewed class distributions with few minority examples.Most contain at most a few thousand examples, limiting machine-learning performance.
- Data Mixing: Combining datasets may mitigate size and domain limitations, but differing task definitions and label inventories complicate integration.Label embeddings are proposed as one way to learn interactions between disjoint but semantically related label sets.
- Multilinguality: Multilinguality matters because claims, evidence, and stances may use different languages, yet only a handful of datasets cover languages beyond English.Cross-lingual training can improve performance through better representations or regularisation.
- Context and Multimodality: Future systems must model user and target context and combine modalities to capture stance around increasingly multimodal mis- and disinformation.Relevant context may include prior activity, interactions, threads, interests, images, videos, and propagation patterns.
- Shades of Truth: Finer-grained truth categories could extend stance beyond binary judgments and represent shades of truth.Existing fact-checking schemes include half-true, mixed, no factual evidence, barely true, and utterly false.
- Label Semantics: Common stance labels do not capture stance strength, and label-definition mismatches can hinder transfer across datasets.More granular labels may improve transfer between datasets.
- Explainability: Explainability is important for mis- and disinformation detection, but existing explanations can depend on gold snippets, misleading attention weights, or noisy topics.FEVER is artificially created and limited to Wikipedia, while other systems provide explanations only to a limited extent.
- Integration: Tighter integration between stance detection and fact-checking could guide human fact-checkers toward relevant evidence, while source credibility can help mitigate vocal-minority risks.Stance appears in tweets, news articles, user posts, Wikipedia sentences, and Wiki tables, with multiple formulations in fact-checking pipelines.
6 Conclusion
The survey reviews stance detection for mis- and disinformation across fake news, rumours, misconceptions, and fact-checking, then distils approaches into lessons and future trends.
- Scope: The survey covers stance detection for fake news, rumour verification, misconception identification, and fact-checking.It also discusses approaches used across these applications.
- Conclusions: The paper summarises observed phenomena as lessons learned and promising future trends.These conclusions follow the survey of the current state of stance detection for mis- and disinformation.
A Examples of Stance
Stance-detection scenarios vary by target, context, document structure, and reasoning demands, with stance used either directly or through aggregated viewpoints.
- Task Settings: Stance targets can be headlines, comments, claims, or topics, while contexts vary by domain and structure.The survey examples include news and Twitter settings.
- News: News scenarios require models to handle long documents and reason across multiple fragments that may express different stances.Summarising an article can simplify this setting.
- Social Media: Social-media stance is often short, informal, and dependent on conversational context, external URLs, or implicit topics.Such text may require normalisation for slurs, emojis, and other informal language.
- Multiple Evidence: Claim verification may require multi-hop reasoning over a chain of evidence when no single passage directly supports or refutes a claim.The evidence chain supplies enough contextual knowledge to assess claim veracity.
- Use in Detection: Stance supports mis- and disinformation detection either directly or as multiple viewpoints aggregated into a final decision.The latter pattern is illustrated for several multi-source and conversational scenarios.
- Datasets: The survey discusses these setups together with publicly available datasets focused on stance in mis- and disinformation identification.The examples are presented as illustrative scenarios rather than a single task formulation.
B Additional Formulations of Stance as a Component for Fact-Checking
Additional formulations use stance to detect misconceptions, retrieve previously fact-checked claims, profile media sources, and interpret varied fact-checking examples.
- Misconceptions: Misconception detection evaluates whether a tweet agrees, disagrees, or has no stance toward known COVID-19 misconceptions.The formulation uses misconceptions listed in Wikipedia.
- Fact-Checked Claims: Previously fact-checked claims can be assessed by comparing their stance toward checked stories, known misconceptions, and facts.This provides another formulation for evaluating dubious content.
- Media Profiling: Media profiling can use stance on controversial topics to identify political leanings of media outlets and influential Twitter users.The cited approach clusters users around vocal core users based on behaviours such as retweeting.
- Stance Labels: Table 3 uses support/for, deny/against, query, and comment to annotate expressed stance in its illustrative scenarios.These labels are applied to examples spanning headlines, evidence, and conversational posts.
C Systems and Applications
The surveyed systems use stance detection within pipelines for rumour verification, credibility assessment, fact-checking, and fake-news detection. Across these applications, stance predictions are combined with retrieved evidence, linguistic or social-context features, and explanations or factuality scores.
- Credibility assessment: CredEye combines predicted article stances toward a claim with linguistic features to produce a credibility score.The system retrieves relevant Web articles and analyzes their credibility in relation to the input claim.
- Fact-checking: A fact-checking prototype combines article stances, source reputation, and annotator reliability to assess claim veracity and provide explanations.The explanations were reported to improve fact-checkers’ overall satisfaction and trust in the predictions.
- Rumour verification: Rumour verification pipelines use stance classification to collect positions toward a rumour before aggregating information for veracity classification.Zubiaga et al. describe a four-step process spanning rumour detection, tracking, stance classification, and veracity classification.
- Fact-checking: FAKTA uses stance predictions and entailed spans from retrieved sources alongside linguistic analysis to generate explanations and a factuality score.Its outputs operate at both document and sentence level.
- Fake-news detection: FANG incorporates user-comment stances toward target articles with temporal, interaction, article–source, and source-reliability information for fake-news detection.The model treats stance as an integral component of its social-context representation.