Source-linked AI summary
Tweet, but Verify: Epistemic Study of Information Verification on Twitter
Arkaitz Zubiaga, Heng Ji
TL;DR
In fast-moving events, tweets can precede other sources and include claims that are difficult to verify, while research had not established how accurately users judge tweet credibility. The paper surveys users’ perceptions of Hurricane Sandy witness pictures using epistemology-informed tweet features and compares their judgments with expert assessments. Author details were associated with more accurate perceptions, whereas repeated corroborating tweets could mislead users about hoaxes.
Problem
The accuracy of users’ credibility judgments about tweets had not been studied, even though fast-moving events can leave users with little opportunity to verify information against other sources.
Source: §1 · p. 1Method
The researchers surveyed users on credibility cues in Hurricane Sandy witness-picture tweets and compared their perceptions with expert assessments.
Source: §1 · p. 2, §3.2 · p. 5Results
Author details led to the most accurate perceptions in terms of precision, while repeated tweets supporting the same information could mislead users into trusting a fake claim.
Source: §4 · p. 10Takeaways & Limitations
Showing additional author details may help readers assess tweet veracity, but author information needs to be interpreted alongside other tweet components.
Source: §4 · p. 10
Abstract
from arXiv · showhide
While Twitter provides an unprecedented opportunity to learn about breaking news and current events as they happen, it often produces skepticism among users as not all the information is accurate but also hoaxes are sometimes spread. While avoiding the diffusion of hoaxes is a major concern during fast-paced events such as natural disasters, the study of how users trust and verify information from tweets in these contexts has received little attention so far. We survey users on credibility perceptions regarding witness pictures posted on Twitter related to Hurricane Sandy. By examining credibility perceptions on features suggested for information verification in the field of Epistemology, we evaluate their accuracy in determining whether pictures were real or fake compared to professional evaluations performed by experts. Our study unveils insight about tweet presentation, as well as features that users should look at when assessing the veracity of tweets in the context of fast-paced events. Some of our main findings include that while author details not readily available on Twitter feeds should be emphasized in order to facilitate verification of tweets, showing multiple tweets corroborating a fact misleads users to trusting what actually is a hoax. We contrast some of the behavioral patterns found on tweets with literature in Psychology research.
1 INTRODUCTION
Twitter’s speed makes it a timely source for breaking events but also allows hoaxes to circulate when information is difficult to verify. This study tests whether credibility cues on tweets help users distinguish real from fake witness pictures, comparing their judgments with expert evaluations.
1 INTRODUCTION
- Breaking news often appears on Twitter before other sources can confirm it, and witness pictures may be difficult to verify in real time.Meanwhile, users continue commenting on and sharing tweets, making presentation relevant to how information is perceived and spread.Source: §1 · p. 1
- Earlier research examined which tweet features influence credibility judgments, but had not established how accurate those judgments are.The study asks whether users’ reliance on cues such as the author helps them judge veracity correctly.Source: §1 · p. 1
- The researchers survey users on separate tweet features and compare their credibility perceptions with final evaluations by professionals.Epistemology research on how people form beliefs informs the survey design.Source: §1 · p. 2
- The study finds that author details can improve credibility judgments, while tweet writing and spelling may harm accuracy and repeated exposure to corroborating hoaxes can mislead users.The survey examined witness pictures from Hurricane Sandy and compared users’ ratings with expert assessments.Source: §1 · p. 2
2 ACCURACY OF INFORMATION AND EPISTEMOLOGY
Fast-moving information can be difficult to verify when tweets precede other sources and provide little opportunity for corroboration. The section introduces epistemology as a framework for examining credibility cues, then applies its four verification features to users’ judgments of Hurricane Sandy witness pictures against expert assessments.
2.1 Accuracy of Information
- Real-time tweets can be difficult to verify when no other source is available, especially for disaster pictures that lack immediate independent evidence.The rapid spread of inaccurate information can have serious consequences, including for health and finances.Source: §2.1 · p. 2
2.2 Epistemology
- Epistemology studies how people form beliefs from acquired knowledge, and credibility judgments aggregate perceptions of an information item’s features.Fallis’s framework identifies authority, plausibility and support, independent corroboration, and presentation as cues for assessing information.Source: §2.2 · pp. 2–3
- The study applies these four features to Twitter by surveying credibility perceptions of Hurricane Sandy witness pictures and comparing them with expert assessments.The pictures were selected as a fast-paced context in which immediate verification was difficult.Source: §2.2 · p. 3
3 EPISTEMIC STUDY OF TWEETS
The study collected Hurricane Sandy witness pictures, surveyed users on credibility cues in tweets, and compared their ratings with professionals’ real-or-fake assessments. Users often trusted hoaxes when claims were repeated, while author information and content cues related differently to their judgments.
3.1 Data Collection
- The study filtered 14.9 million Sandy tweets to 332 popular pictures from English-source tweets, later labeled 216 real and 116 fake by professional assessments.Pictures had at least 100 associated tweets; professionals assessed context, prior image uses, and possible manipulation.Source: §3.1 · pp. 3–4
- Fake pictures were at least as popular as real ones, with higher median and average shares, so propagation did not track truth.The authors note that sharing could reflect humor or questioning as well as belief in a picture.Source: §3.1 · p. 4, Table 1 · p. 4
3.2 Epistemology of Tweets
- AMT workers separately rated authority, text and picture plausibility, independent corroboration, and presentation, then rated whole tweets as shown on Twitter.The feature ratings isolated parts of tweets before a complementary whole-tweet task.Source: §3.2 · pp. 5–7
3.3 Results
- Ratings were higher for real pictures across all six measures, but fake pictures still received high ratings and real pictures sometimes received low ones.The median text-plausibility rating for fake pictures was much higher, indicating that many hoax tweets sounded plausible.Source: §3.3 · pp. 7–8, Figure 2 · p. 8
- Whole-tweet ratings recovered the most real pictures, yet more than 16% of tweets rated real were fake; author-only ratings were slightly more precise.Of 39 whole-tweet false positives, 31 (79.5%) were correctly identified as fake from the author alone.Source: §3.3 · p. 8, Table 2 · p. 9
- Workers rated 71% of fake tweets higher for corroboration than for text plausibility, and repeated claims often misled users into trusting hoaxes.Corroboration ratings were slightly less accurate than text-plausibility ratings.Source: §3.3 · pp. 8–9
- Whole-tweet ratings correlated more with corroboration and plausibility than with authority; presentation had even lower correlation, indicating users favored content over authors.The reported Pearson correlations were statistically significant (p < .0001).Source: §3.3 · p. 9, Figure 3 · p. 9
4 DISCUSSION
The discussion argues that author details can improve tweet verification, while repeated corroborating tweets can mislead users about hoaxes. It also recommends richer verification interfaces and user instruction, while noting that feature-by-feature ratings do not capture how tweet components work together.
4 DISCUSSION
- Feature-by-feature ratings isolate each cue from influence by the rest of the tweet, although the authors caution that real assessments require analyzing features together.The study compared users’ credibility perceptions with picture veracity determined by professionals.Source: §4 · p. 10
- 31 of 39 fake tweets mistakenly deemed real from the picture alone would have been correctly identified as fake using author details (79.5%).The authors say author details were the feature associated with the most accurate perceptions in terms of precision, and that details should be combined with other tweet components.Source: §4 · p. 10
- Repeated tweets supporting the same claim can mislead users into trusting a hoax, while spelling and grammar were not useful for verification.Typos and capitalization issues may reflect mobile posting rather than false content.Source: §4 · p. 10
- Verification interfaces could show a user reliability score and strengthen location and corroboration-independence cues, but these features need further research.Follower counts alone do not establish reliability, since parody accounts may also have followers.Source: §4 · p. 10
- Because users may rarely investigate sources in Twitter’s fast stream, the authors recommend instruction on accuracy cues alongside showing more user details.Users should also be made aware of the problem of repeated exposure to fake information.Source: §4 · p. 10
5 RELATED WORK
Prior Twitter-credibility research includes automated assessment and studies of users’ perceptions, but earlier perception work examined how features shape ratings rather than whether those perceptions lead to correct verification. This study addresses that complementary question.
5 RELATED WORK
- Automated Twitter-credibility work has used classifiers and ranking systems based on predefined features, graphs, external sources, and tweet content.A classifier using message, user, topic, and propagation features was reported as highly accurate against AMT assessments.Source: §5 · pp. 10–11
- Other work addressed online-content credibility, including blog users’ comparisons with traditional media and journalists’ challenges verifying real-time social-media information.Source: §5 · p. 11
- Earlier user-perception studies examined how tweet alterations affect credibility ratings and compared perceptions across U.S. Twitter and Chinese Weibo users.The first found that major interfaces lacked sufficient credibility information and that more author details could help.Source: §5 · p. 11
- This study complements work on which tweet features shape perceptions by examining whether those perceptions help users correctly distinguish truthful tweets from hoaxes.Source: §5 · p. 11
6 CONCLUSION
The paper studies whether credibility cues help users assess tweet veracity during a fast-paced event, using Hurricane Sandy pictures and U.S. AMT respondents. It concludes that additional tweet features may aid verification and identifies complementary approaches for future work.
6 CONCLUSION
- The study surveyed U.S. AMT workers about Hurricane Sandy picture tweets, assessing credibility cues against professionals’ evaluations of picture veracity.The authors describe it as the first epistemic study of tweets and say it examined how features suggested in epistemology relate to accurate or inaccurate information judgments.Source: §6 · p. 11
- The authors identify journalist verification guidelines and computational approaches using user proximity and content similarity in Twitter’s social graph as future directions.Source: §6 · p. 11