Source-linked AI summary
Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads
Arkaitz Zubiaga, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, Peter Tolmie
TL;DR
Breaking-news rumours spread through social media, yet their life-cycle dynamics and users’ responses remain insufficiently understood. The paper develops a journalist-supported annotation methodology and analyses 330 rumour threads comprising 4,842 tweets from 9 newsworthy events. True rumours are resolved faster than false ones, while users generally support unverified rumours and reputable users may post certain, evidence-backed information that remains unverified.
Problem
The life-cycle dynamics of social-media rumours, including how users spread, support, and deny them before and after veracity resolution, remain insufficiently understood.
Method
The paper develops a methodology for journalists to collect, identify, and annotate rumour conversations, then analyses 330 threads comprising 4,842 tweets from 9 newsworthy events.
Results
True rumours are typically resolved within 2 hours, whereas false rumours take about 14 hours to be debunked; users also tend to support unverified rumours.
Takeaways & Limitations
Users appear less able to distinguish true from false rumours while veracity remains unresolved, reinforcing the value of machine-learning assistance for real-time rumour-veracity assessment.
Takeaways & Limitations
Some rumour resolutions may occur outside the data-collection timeframe or outside Twitter, constraining what the dataset captures.
Abstract
from arXiv · showhide
As breaking news unfolds people increasingly rely on social media to stay abreast of the latest updates. The use of social media in such situations comes with the caveat that new information being released piecemeal may encourage rumours, many of which remain unverified long after their point of release. Little is known, however, about the dynamics of the life cycle of a social media rumour. In this paper we present a methodology that has enabled us to collect, identify and annotate a dataset of 330 rumour threads (4,842 tweets) associated with 9 newsworthy events. We analyse this dataset to understand how users spread, support, or deny rumours that are later proven true or false, by distinguishing two levels of status in a rumour life cycle i.e., before and after its veracity status is resolved. The identification of rumours associated with each event, as well as the tweet that resolved each rumour as true or false, was performed by a team of journalists who tracked the events in real time. Our study shows that rumours that are ultimately proven true tend to be resolved faster than those that turn out to be false. Whilst one can readily see users denying rumours once they have been debunked, users appear to be less capable of distinguishing true from false rumours when their veracity remains in question. In fact, we show that the prevalent tendency for users is to support every unverified rumour. We also analyse the role of different types of users, finding that highly reputable users such as news organisations endeavour to post well-grounded statements, which appear to be certain and accompanied by evidence. Nevertheless, these often prove to be unverified pieces of information that give rise to false rumours. Our study reinforces the need for developing robust machine learning techniques that can provide assistance for assessing the veracity of rumours.
Introduction
Social media broadens access to breaking-news information but also accelerates the circulation of unverified rumours. This study addresses limited understanding of rumour propagation by analysing how users spread, support, and deny rumours across their life cycles.
- Social media lets journalists, news organisations, and ordinary citizens share diverse breaking-news information, while requiring users to assess source accuracy.Information can appear on social media before mainstream outlets, but users must sift through sources to evaluate it.
- False rumours can influence public understanding and produce harmful social consequences, including disruption to financial markets.A fabricated report that the White House had been bombed temporarily spooked US stock markets.
- The paper addresses limited research on rumour propagation by examining Twitter conversations before and after rumours’ veracity is resolved.Its methodology supports analysis of how rumours are spawned, confirmed, debunked, supported, and denied across nine newsworthy events.
- The study defines rumours as circulating information of questionable veracity that appears credible, is difficult to verify, and motivates efforts to establish the truth.This operational definition allows rumours to be ultimately true, partly false, or entirely false.
- Earlier social-media research often examined diffusion or already-debunked false rumours, leaving conversational responses and changing veracity status less explored.The paper therefore analyses diffusion, support, and denial across a broad multi-event dataset rather than restricting attention to false rumours.
- The contribution combines a conversational annotation scheme, a crowdsourced dataset of 330 threads from 9 events, and quantitative analysis of rumour diffusion, support, and denial.The annotation captures observable conversational features, while the analysis follows rumours from release through eventual resolution.
Materials and Methods
The study builds a journalist-guided, crowdsourced annotation methodology for collecting and analysing rumour conversations across nine newsworthy events. It organises rumours into stories and threads, annotates conversational features, and evaluates the resulting dataset and annotation agreement.
- Data collection: Twitter’s streaming API collected tweets from five breaking-news events and four specific rumours identified a priori.
- Journalist annotation: Journalists identified rumourous source tweets, grouped them into stories, assigned veracity status, and marked a decisive resolving tweet when available.
- Annotation scheme: Each conversation was annotated for support or response type, certainty, and evidentiality across source and direct or nested response tweets.
- Dataset: 330 conversations were categorised into 140 stories, comprising 159 true, 68 false, and 103 unverified conversations.
- Annotation quality: 62.3% overall inter-annotator agreement was achieved, with lower agreement for response tweets and slightly greater difficulty annotating evidentiality.
Results
Across nine events, rumours began unverified and were resolved at different speeds, with true rumours resolved much sooner than false ones. Before resolution, users generally supported rumours, while after resolution discussion and denial patterns changed substantially, especially for false rumours.
- 103 rumours remained unverified, 159 were later proven true, and 68 were found false.
- True rumours had a median resolution time of about 2 hours, whereas false rumours took over 14 hours.False rumours were often unresolved within the first 10 hours and, when eventually resolved, frequently took 15–20 hours.
- Early tweets supporting still-unverified rumours generated the most retweets, while denial tweets were retweeted least often.Among pre-supporting tweets, those attached to ultimately true rumours were retweeted more than those attached to false rumours.
- Before resolution, true rumours had a median support ratio of 0.156 versus 0.067 for false rumours, but both groups were predominantly supported.The small difference suggests that collective responses were not sufficient to determine veracity before resolution; unverified stories had a median support ratio of 0.04348.
- Only false rumours received more denials than support after the resolving tweet; otherwise, support generally exceeded denial or neither response occurred.
- After resolution, discussion increased, particularly for false rumours, while support ratios declined and false-rumour discussion shifted toward denial.Certainty changed little for true rumours, whereas evidentiality dropped significantly for false rumours from a median of 0.8944 before resolution to 0.667 afterward.
Discussion and Conclusions
Across 330 rumour threads and 4,842 tweets from nine newsworthy events, the study finds that users spread and support unverified rumours before resolution, while responses change after verification or debunking. True rumours resolve faster than false ones, and even reputable news organisations may circulate ultimately false rumours despite using confident, evidence-linked posts.
- 330 rumour threads comprising 4,842 tweets across 9 newsworthy events enabled analysis of diffusion, support, denial, certainty, evidentiality, and user characteristics.
- True rumours are usually corroborated within 2 hours, whereas false rumours take about 14 hours to be debunked.
- Unverified rumours generate a sharp retweet burst in the first few minutes, while interest in spreading them declines after veracity is resolved, especially for false rumours.
- Discussion increases after resolution because denials rise and support declines, regardless of whether the rumour is confirmed or debunked.
- Certainty changes little across veracity statuses, but tweets contain significantly less evidence after false rumours are debunked than while they remain unverified.
- News organisations tend to post certain, evidence-linked statements, yet these can still be unverified reports that later prove false, reflecting pressure to publish.