PaperlayerView exact revision on arXiv

Source-linked AI summary

Modeling Conversation Structure and Temporal Dynamics for Jointly Predicting Rumor Stance and Veracity

Penghui Wei, Nan Xu, Wenji Mao

arXiv:1909.08211v1cs.CLcs.SI

TL;DR

False rumors can harm the public, and public stances toward rumors offer clues to their veracity. The paper jointly predicts stance and veracity using conversation-structure modeling followed by temporal stance modeling, and reports better performance than existing methods on both tasks.

  • Problem

    Automatic false-rumor detection matters because rumors can cause harm and manual debunking of massive online content is time-consuming.

    Source: §1 · p. 1

  • Method

    A hierarchical multi-task model uses graph convolution to classify tweet stances from conversation structure, then models stance evolution over time for rumor veracity prediction.

    Source: §1 · p. 2

  • Results

    On two benchmark datasets, the framework outperforms existing methods in both rumor stance classification and veracity prediction.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The framework combines conversation structure for stance classification with temporal stance evolution for veracity prediction in a jointly trained model.

    Source: §1 · p. 2

    Conversation reply relationships are treated as undirected edges in the tree.

    Source: §4.1 · p. 3

Abstract

from arXiv · show

Automatically verifying rumorous information has become an important and challenging task in natural language processing and social media analytics. Previous studies reveal that people's stances towards rumorous messages can provide indicative clues for identifying the veracity of rumors, and thus determining the stances of public reactions is a crucial preceding step for rumor veracity prediction. In this paper, we propose a hierarchical multi-task learning framework for jointly predicting rumor stance and veracity on Twitter, which consists of two components. The bottom component of our framework classifies the stances of tweets in a conversation discussing a rumor via modeling the structural property based on a novel graph convolutional network. The top component predicts the rumor veracity by exploiting the temporal dynamics of stance evolution. Experimental results on two benchmark datasets show that our method outperforms previous methods in both rumor stance classification and veracity prediction.

1 Introduction

False rumors can cause public harm, motivating automatic detection, while public stance provides clues to rumor veracity. The paper proposes a jointly trained hierarchy that models conversation structure for stance classification and stance evolution for veracity prediction.

1 Introduction

  • Automatic false-rumor detection is needed because rumors can cause harm and manually debunking massive online content is time-consuming.Source: §1 · p. 1
  • Public stances toward rumors provide indicative clues for veracity prediction, making stance classification a preceding task.Source: §1 · p. 1
  • Conversational-GCN learns tweet stance features by aggregating information from structurally neighboring tweets, with nearer neighbors considered more informative.Source: §1 · pp. 1–2
  • A top-level RNN models temporal stance evolution for veracity prediction, using content and stance features learned by the bottom component.Source: §1 · p. 2
  • The hierarchical multi-task framework jointly learns stance and veracity, exploiting their interrelation rather than treating the tasks separately.Source: §1 · p. 2
  • On two benchmark datasets, the hierarchical framework outperforms existing methods in both rumor stance classification and veracity prediction.Source: §1 · p. 2

2 Related Work

Prior work studies rumor stance through linguistic, sequential, temporal, and other conversation-based methods, and rumor veracity through varied features and models. This paper distinguishes its approach by hierarchically integrating structural stance modeling with temporal stance evolution.

2 Related Work

  • Twitter stance methods include sequential and temporal conversation models, feature-based approaches, label propagation, and methods using linguistic or point-process features.Source: §2 · pp. 2–3
  • Veracity research uses linguistic, time-series, and propagation features, with neural models also capturing time series and propagation paths.Source: §2 · p. 3
  • Prior studies use stance labels or HMMs for veracity prediction, whereas this method learns stance-evolution features with an RNN and jointly predicts both tasks.Source: §2 · p. 3
  • Unlike parallel multi-task models with shared representations, this method hierarchically learns structural stance features and models stance evolution to integrate the tasks.Source: §2 · p. 3

3 Problem Definition

The paper represents each rumor discussion as a tree rooted at its source tweet, with replies linked by direct or indirect response relationships. It predicts each tweet’s stance and the rumor’s veracity as separate tasks.

3 Problem Definition

  • A conversation thread consists of a source tweet and direct or indirect replies, with replying relationships forming a tree rooted at the source.Source: §3 · p. 3
  • Stance classification assigns each conversation tweet one of four labels, while veracity prediction assigns the rumor one of three labels.The stance labels are supporting, denying, querying, and commenting; veracity labels are true, false, and unverified.Source: §3 · p. 3

4 Proposed Method

Hierarchical-PSV links conversation-structure modeling for tweet stance classification to temporal modeling of stance evolution for rumor veracity prediction. Its Conversational-GCN and Stance-Aware RNN are jointly trained to integrate the two tasks’ feature learning.

4 Proposed Method

  • Hierarchical-PSV uses a bottom Conversational-GCN for tweet stance features and a top Stance-Aware RNN for rumor veracity prediction.The top component uses features learned by the bottom component to model stance evolution.Source: §4 · p. 3

4.1 Conversational-GCN: Aggregation-based Structure Modeling for Stance Prediction

  • Conversational-GCN aggregates neighboring tweets’ features to predict stances using conversation-customized graph convolution.Its modified operation addresses deep-tree receptive-field limits and weakened self-information: multiplication expands the receptive field, while an identity matrix elevates the target tweet’s importance.Source: §4.1 · p. 4

4.2 Stance-Aware RNN: Temporal Dynamics Modeling for Veracity Prediction

  • Stance-Aware RNN combines content and stance features in chronological order, models their evolution with a GRU, then max-pools the sequence for veracity prediction.A feed-forward network with softmax produces the predicted veracity distribution from the pooled vector.Source: §4.2 · p. 5

4.3 Jointly Learning Two Tasks

  • The framework jointly optimizes stance and veracity losses, with a trade-off parameter, to integrate feature learning across the two related tasks.Conversational-GCN learns content and stance features, which Stance-Aware RNN uses to model temporal evolution for veracity prediction.Source: §4.3 · p. 5

5 Experiments

Experiments evaluate stance classification and veracity prediction on two benchmark datasets. The results compare the proposed models with prior methods and examine the contribution of joint learning.

5.1 Data & Evaluation Metric

  • The evaluation uses SemEval-2017 task 8 for stance and veracity, and PHEME for veracity prediction with leave-one-event-out cross-validation.SemEval contains 325 rumor threads; PHEME contains 2,402 conversations, and macro-averaged F1 is used for both tasks because of class imbalance.Source: §5.1 · pp. 5–6

5.2 Implementation Details

  • The experiments use two GCN layers, with the number of layers fixed at L = 2.Implementation details are provided in Appendix A.Source: §5.2 · p. 6

5.3.1 Results: Rumor Stance Classification

  • Conversational-GCN achieves better macro-averaged F1 than all stance-classification baselines, with stronger denying and querying predictions than the other methods.The table reports classwise F1 scores for supporting, denying, querying, and commenting stances.Source: §5.3.1 · p. 6, Table 2 · p. 6

5.3.2 Results: Rumor Veracity Prediction

  • Joint methods outperform the BranchLSTM+NileTMRG pipeline, Hierarchical-PSV outperforms the parallel MTL2 model, and stance supervision improves over Hierarchical GCN-RNN.For a batch of 32 conversations, Hierarchical GCN-RNN takes 0.18 seconds per training step versus 5.02 seconds for TD-RvNN.Source: §5.3.2 · p. 7, Table 3 · p. 7

5.4 Further Analysis and Discussions

Further analyses test graph convolution across conversation depths, ablate stance and temporal features, vary the stance-loss weight, and inspect one false-rumor prediction.

5.4.1 Effect of Customized Graph Convolution

  • Conversational-GCN outperforms Original-GCN and BranchLSTM at most conversation depths, with an advantage for identifying deep-thread stances.The comparison reports macro-averaged F1 for tweets grouped by thread depth.Source: §5.4.1 · p. 8, Figure 4 · p. 8

5.4.2 Ablation Tests

  • Removing stance features or replacing or removing the GRU block hurts veracity prediction, supporting the use of stance-aware temporal modeling.The stance-feature ablation inputs only tweet content to the top RNN; the table reports ablations on PHEME.Source: §5.4.2 · p. 8, Table 4 · p. 8

5.4.3 Interrelation of Stance and Veracity

  • As λ increases from 0.0 to 1.0, false- and unverified-rumor identification generally improves, linking stronger stance supervision with more accurate veracity clues.Figure 5 reports false-class and unverified-class F1 scores on PHEME.Source: §5.4.3 · p. 8, Figure 5 · p. 8

5.5 Case Study

  • In the illustrated false-rumor conversation, replies shift from supporting to querying to denying, and the model predicts veracity correctly.The case study also shows max-pooling selecting informative tweets in the conversation.Source: §5.5 · pp. 8–9, Figure 6 · p. 8

6 Conclusion

The paper concludes that its hierarchical multi-task framework jointly predicts rumor stance and veracity on Twitter. It reports effective handling of deep conversation structures and stronger performance than existing methods.

6 Conclusion

  • The framework pairs Conversational-GCN for structure-aware stance classification with Stance-Aware RNN for modeling stance evolution in veracity prediction.The two components are arranged hierarchically and jointly trained.Source: §6 · p. 9
  • The experiments support effective handling of deep conversation structures and better overall performance than existing methods.Source: §6 · p. 9
  • Future work will explore external context, multilingual scenarios, and rumor diffusion from a social-science perspective.Source: §6 · p. 9

A.1 Hyperparameters

The appendix specifies training and model settings for SemEval-2017 task 8 and PHEME.

A.1 Hyperparameters

  • For SemEval, the model uses fixed 200-dimensional Skip-gram embeddings, a two-layer GCN with output sizes 200 and 4, and λ = 1.The content feature dimension is 200, and dropout is 0.5 on all but the last GCN layer and the veracity FNN layer.Source: §A.1 · p. 11
  • SemEval training uses Adam, a 0.001 learning rate, and batch size 32; after development-set tuning, training and development data are merged for retraining.Source: §A.1 · p. 11
  • PHEME uses a 0.005 learning rate, and conversations without stance labels do not contribute to the stance-classification loss.Other configurations follow those used for SemEval.Source: §A.1 · p. 11

A.2 The CNN Layer in Ablation Study

The ablation replaces the GRU with a CNN that captures only local temporal information, then uses the CNN output to predict rumor veracity.

A.2 The CNN Layer in Ablation Study

  • The ablation replaces the GRU layer with a CNN layer that captures only local temporal information.The stated purpose is to demonstrate the effectiveness of modeling temporal dynamics of stance evolution.Source: §A.2 · p. 11
  • The CNN uses a 1D convolution and max-pooling, with filter windows of 2, 3, and 4 and 100 feature maps per window.Source: §A.2 · p. 11
  • The CNN output vector feeds an FNN with softmax to produce a predicted veracity distribution.Source: §A.2 · p. 11

B Numerical Results of Figure 4

Table 5 gives numerical results associated with Figure 4; its caption identifies stance classification results across different depths.

B Numerical Results of Figure 4

Loading 1909.08211v1…