PaperlayerView exact revision on arXiv

Source-linked AI summary

False Information on Web and Social Media: A Survey

Srijan Kumar, Neil Shah

arXiv:1804.08559v1cs.SIcs.CLcs.CYcs.DL

TL;DR

False information can spread rapidly online and has affected real-world events, making its proliferation and impact important to understand. This survey synthesizes research on mechanisms, rationale, impact, characteristics, and detection across fake reviews, hoaxes, and fake news; it reports that a small fraction of false stories attract high engagement and deeper reshare cascades than true information.

  • Problem

    Characterizing how false information spreads and deceives readers matters because online platforms enable rapid distribution and false information has affected markets, disaster responses, and terrorist attacks.

    Source: §1 · p. 1

  • Method

    The survey reviews mechanisms, rationale, impact, characteristics, and detection of fake reviews, hoaxes, and fake news.

    Source: §7 · p. 29

  • Results

    A small fraction of false-information stories are highly impactful, while fake news is spread by very few users and spreads rapidly at initial release.

    Source: §1 · pp. 2–3

  • Takeaways & Limitations

    The paper provides a comprehensive overview of false information on the web and social media.

    Source: §1 · p. 3

    Standardized comparisons of existing algorithms on Buzzfeed, LIAR [107], CREDBANK, and FakeNewsNet [89] [90] had not been conducted.

    Source: §7 · p. 30

Abstract

from arXiv · show

False information can be created and spread easily through the web and social media platforms, resulting in widespread real-world impact. Characterizing how false information proliferates on social platforms and why it succeeds in deceiving readers are critical to develop efficient detection algorithms and tools for early detection. A recent surge of research in this area has aimed to address the key issues using methods based on feature engineering, graph mining, and information modeling. Majority of the research has primarily focused on two broad categories of false information: opinion-based (e.g., fake reviews), and fact-based (e.g., false news and hoaxes). Therefore, in this work, we present a comprehensive survey spanning diverse aspects of false information, namely (i) the actors involved in spreading false information, (ii) rationale behind successfully deceiving readers, (iii) quantifying the impact of false information, (iv) measuring its characteristics across different dimensions, and finally, (iv) algorithms developed to detect false information. In doing so, we create a unified framework to describe these recent methods and highlight a number of important directions for future research.

1 INTRODUCTION

The survey synthesizes research on public false information across the web and social media, covering who spreads it, why readers are deceived, its impact and characteristics, and detection methods.

Survey scope

  • The survey covers fake reviews, collaborative-platform hoaxes, and social-media fake news, distinguishing misinformation from disinformation and opinion-based from fact-based content.Its scope is public false information aimed at many people, rather than targeted scams.Source: §1 · p. 2

Actors

  • Bots and sockpuppets amplify false information through mass sharing, mutual following, and coordinated agreement that can create an illusion of public consensus [47].These fake or compromised accounts can occupy central network positions and appear trustworthy.Source: §1 · p. 2

Rationale for deception

  • Across experiments, people identified false information with only 53%–78% accuracy, while polished, long, well-referenced content, echo chambers, and reader biases contribute to deception.Low education and media consumption are also cited as factors that make readers more susceptible.Source: §1 · p. 2

Impact

  • A small fraction of false stories attracts high engagement, generates deeper reshare cascades than true information, persists longer, and spreads across the web.Studies measure impact using likes, reshares, and pre-removal lifetime.Source: §1 · p. 2

Characteristics

  • False information differs from real information in content, timing, users, networks, and spread; fake reviews tend to be longer and more exaggerated, while fake news spreads rapidly at first.Fake reviews also appear in short bursts from relatively new accounts, and most fake news is spread by very few users.Source: §1 · pp. 2–3

Detection

  • Detection methods use content and behavior features, network structure, or propagation models, with reported accuracies reaching the high eighties and nineties on real-world datasets.The survey groups these approaches as feature-based, graph-based, and propagation-modeling algorithms.Source: §1 · p. 3

2 TYPES OF FALSE INFORMATION

The survey classifies false information along two dimensions: authorial intent (misinformation or disinformation) and knowledge content (opinion-based or fact-based).

2.1 Categorization based on intent

  • Misinformation is shared without intent to deceive, often through distortions of true information caused by misunderstanding, inattention, or cognitive bias; differing interpretations can also shape its spread.Source: §2.1 · p. 4
  • Disinformation is intentionally deceptive, often aiming to sway public opinion or drive traffic to ad-supported websites; the survey focuses mainly on its technological aspects.The survey cites political disinformation during the 2016 U.S. presidential election [29] [38] [87], which even led to public shootings.Source: §2.1 · p. 4

2.2 Categorization based on knowledge

  • Opinion-based false information lacks absolute ground truth, whereas fact-based information contradicts, fabricates, or conflates single-valued ground truth.Opinion-based examples include fake product reviews; fact-based examples include fake news, rumors, and fabricated hoaxes. Fact-based false information aims to make the false version harder to distinguish from the true one.Source: §2.2 · pp. 4–5

3 ACTORS AND RATIONALE OF SUCCESSFUL DECEPTION BY FALSE INFORMATION

Coordinated fake accounts can amplify false information, but human readers also spread it and are often deceived. The section describes how presentation, echo chambers, and cognitive and social biases make false information persuasive.

3.1 Bad actors: bots and sockpuppets

  • Deceptive sockpuppets were twice as common as benign ones in one study, and coordinated accounts can fabricate agreement while occupying central network positions.Sybil accounts also integrate into networks to evade detection and increase their influence.Source: §3.1 · p. 6
  • Almost one-fifth of Twitter political chatter came from bots, and another study found bots generated 25% of false-information tweets.Botnets rapidly broadcast messages, inflate users’ apparent social status, and can target influential users who may reshare false content; fake-follower schemes use distinct account-network structures.Source: §3.1 · p. 6
  • After analyzing 126,000 Twitter false-information cascades, Vosoughi et al. found humans—not bots—spread false information, which still traveled farther, deeper, faster, and broader than true information.Bots accelerated true and false information roughly equally; human false-information spreaders were newer, had fewer followers and followees, and were less active, though other platforms may differ.Source: §3.1 · p. 6

3.2.1 Human inability to discern false information.

  • Humans identified Wikipedia hoax articles correctly only 66% of the time, just above the 50% random-guessing baseline.Readers and even trained Wikipedia patrollers mistook long, well-referenced hoaxes for true articles, showing how genuine-seeming presentation can mislead.Source: §3.2.1 · p. 7
  • Human accuracy was 53.1%–61.9% for fake reviews, while fake-news detection accuracy was 70.5% for crowdsourced news and 78.5% for celebrity news.For machine-generated fake reviews, workers achieved 40.6% precision and 16.2% recall; repetitive errors were easier to spot than minor spelling or grammar mistakes. Across the studies, carefully crafted false information could deceive readers [70, 74, 114].Source: §3.2.1 · p. 8

3.2.2 Formation of echo-chambers.

  • Echo chambers expose users mainly to belief-aligned content and can lower critical fact-checking; repeated exposure to the same false information increases participants’ perceived accuracy linearly, irrespective of credibility.Users clicked significantly less diverse sources on social media than in general search engines; echo chambers can polarize viewpoints through confirmation bias.Source: §3.2.2 · p. 9

3.2.3 Other reasons of successful deception.

  • Confirmation bias, naive realism, and peer norms can make consumers favor belief-aligned false information, while factual corrections can further polarize groups and increase misperceptions.Analytic thinking correlates positively with discernment; demographics associated with more accurate perceptions include higher education, older age, and more time consuming media. Misunderstanding a publisher’s intentions can also create misinformation.Source: §3.2.3 · pp. 9–10

4 IMPACT OF FALSE INFORMATION

The section measures false information’s impact through online engagement and documents both real-world harms and broad, rapid spread. Across studies, most false items were ineffective, but a small fraction was highly impactful.

Impact of False Information

  • False information has an average 12-hour head start before debunking, and rumors spread rapidly at the outset, when unverified claims can go viral.Rumors with the possibility of being true sometimes begin spreading through reputed news organizations.Source: §4 · p. 10
  • 90% of Wikipedia hoaxes were flagged within an hour, while about 1% survived over a year; hoaxes averaged 3 daily views versus 3.5 for non-hoaxes.About 1% of hoaxes drew at least 100 views per day; 7% had at least 5 distinct links clicked from across the web, and each hoax averaged 1.1 such links.Source: §4 · pp. 10–11
  • The top 20 fake 2016 US election stories drew 8,711,000 Facebook engagements versus 7,367,000 for the top 20 true stories; Sandy fake images were shared almost twice as often.The election comparison covered only top-performing stories, so all-story results might differ.Source: §4 · p. 11
  • False Facebook cascades were deeper overall: 3% reached at least 10 reshares versus less than 1% of reference cascades, despite references leading near the original posts.The study identified 16,672 cascades containing 62,497,651 shares; false cascades had more reshares at greater depths, while reference cascades died sooner.Source: §4 · p. 11
  • False tweets spread farther, faster, deeper, and more broadly than true tweets across topics; the top 1% reached over 1,000 users, and false information reached equal audiences six times faster.The pattern held across politics, urban legends, science, business, and other categories; peer-to-peer sharing helped drive its virality.Source: §4 · p. 11
  • Web impact is commonly measured through views, shares, reach, and survival; most false information is ineffective, but a small fraction (typically 1%) is highly impactful.Reported real-world effects include stock-market impact [17], hampered disaster response, and terroristic activity.Source: §4 · pp. 10–12

5 CHARACTERISTICS OF FALSE INFORMATION

The section surveys characteristics that can serve as tell-tale signs distinguishing false from true information, discussing opinion-based and fact-based false information separately and organizing studied features into broad categories.

5 CHARACTERISTICS OF FALSE INFORMATION

  • The section presents characteristics of false information as tell-tale signs for discerning it from true information.The characteristics include textual content, time, ratings, graph structure, and creator properties.Source: §5 · pp. 12–13
  • The discussion treats characteristics of opinion-based and fact-based false information separately across the described axes.Source: §5 · p. 13
  • Table 2 categorizes studied features as text, user, graph, rating score, time, and propagation-based features.Source: §5 · p. 13

5.1 Opinion-based false information

The survey organizes fake-review evidence across textual, rating, temporal, graph, and other properties, and describes recurring patterns of extremity, burstiness, and coordination.

5.1.1 Textual characteristics.

  • Fake reviews are often shorter, more exaggerated and sentimental, less readable, and more focused on aspects external to the venue; fake negative reviews intensify negative emotion and may avoid pronouns.Jindal et al. found many exact duplicate reviews were posted by the same user across different products; 6% of multi-review reviewers had maximum similarity of 1. Sandulescu et al. noted that fraudsters may instead use semantically similar wording.Source: §5.1.1 · pp. 13–14

5.1.2 Ratings characteristics

  • Prior work [36] finds that fake-review writers often skew toward extreme ratings, while Kumar et al. report ratings that diverge from product averages.Some fraudsters give only positive ratings to inflate products or only negative ratings to slander competitors; they may also give high ratings to products rated negatively by fair users. Figure 7 contrasts typical J-shaped aggregate ratings with biased fraudster distributions.Source: §5.1.3 · p. 14, Figure 7 · p. 14

5.1.3 Temporal characteristics.

  • Fraudulent reviewers often post in short bursts, with seconds- or minutes-long interarrival times; reviews at far-apart locations within infeasibly short intervals also signal fraud.Both fraudulent and non-fraudulent users show bimodal posting rates, but fraudulent users are more bursty.Source: §5.1.3 · pp. 14–15

5.1.4 Graph-based characteristics.

  • Overall, opinion-based false information tends to be shorter, more exaggerated, more extremely rated, burstier, and produced in a coordinated “lockstep” fashion.Graph studies associate lockstep behavior with dense subgraphs. Fraudster groups show larger reviewer-to-product group ratios, group sizes, and support counts, while temporally coherent bipartite cores are suggestive of fraud.Source: §5.1.4 · p. 15

5.2 Characteristics of fact-based false information

This section characterizes fact-based false information—hoaxes, rumors, and fake news with a single-valued truth—through its textual, creator, network, propagation, and debunking patterns.

5.2.1 Textual characteristics.

  • Horne and Adali [37] found fake-news titles longer and more emphatic and bodies shorter, repetitive, less informative, and easier to read than real news.Perez et al. [74] also found more social and positive words, greater certainty, and more verbs and time words focused on present and future actions. Kumar et al. found hoaxes longer but with fewer references than non-hoaxes. Discussions of false information show more confusion, disbelief, and hedging; positive sentiment words can signal low credibility because they include mockery.Source: §5.2 · pp. 16–17

5.2.2 User characteristics.

  • Hoaxes come from newer, less-experienced accounts, and similar editors also create non-hoaxes mistaken for hoaxes; bots generated about one-fifth of election-rumor content.Rumor-spreading bots acted in short bursts.Source: §5.2.2 · p. 17

5.2.3 Network characteristics.

  • Bot spreaders cluster in follower networks and become more central in higher k-cores, while alternate-media domains form tightly connected clusters and hoax references are less connected.Kumar et al. found hoaxes had significantly lower local hyperlink clustering coefficients than non-hoax articles, suggesting their references served appearance rather than substantiation.Source: §5.2.3 · pp. 17–18

5.2.4 Propagation characteristics.

  • Rumors can reach 45.6 million of 51.2 million members in eight rounds; false information appears across platforms more often than true information (18% vs 11%) and spreads faster between them.The most common cross-platform route was Reddit to Twitter to 4chan.Source: §5.2.4 · pp. 18–19
  • False information spreads farther, deeper, faster, and broader than true information, especially early, while a handful of users drive much of its spread.Tweets supporting unverified claims can attract sudden bursts of retweets before fact-checking; interest falls substantially after verification.Source: §5.2.4 · pp. 18–19

5.2.5 Debunking characteristics.

  • Debunking lags: false information takes about 14 hours to resolve, and fact-checking arrives 10–20 hours after rumor starts; linked corrections raise deletion probability 4.4 times.After debunking, more tweets deny rumors than support them; replies to false-information tweets express fear, disgust, and surprise, unlike replies to true information.Source: §5.2.5 · pp. 19–20

6 DETECTION OF FALSE INFORMATION

This section surveys false-information detection approaches for opinion-based reviews and fact-based rumors, hoaxes, and news, including feature engineering, graph analysis, and propagation modeling. It also describes how propagation models support detection and mitigation, and how network structure, user behavior, and intervention timing shape rumor persistence and detection performance.

6 DETECTION OF FALSE INFORMATION

  • False information is typically under 10% of instances, is designed to resemble truth, and is difficult to label manually, creating core detection challenges.Labels commonly come from experts, trained volunteers, or Mechanical Turk workers, requiring substantial effort and risking missed misinformation.Source: §6 · p. 21

6.1 Detection of opinion-based false information

  • Opinion-based detection combines text, graph, and time-based methods; text models use supervised features, graph methods jointly estimate user, review, and product trustworthiness, and temporal methods model review sequences.Text features are often combined with user, graph, score, and time features.Source: §6.1 · p. 21

6.1.2 Graph-based detection.

  • Review-graph methods propagate beliefs using homophily between honest users and products, while reported precision was 49% among the 100 users with lowest trustiness.The SpEagle algorithm combines belief propagation with node and edge features and averaged about 0.78 AUC across three Yelp datasets; fraudulent-user detection may miss fake reviews or flag genuine ones.Source: §6.1.2 · pp. 23–24

6.1.3 Detection with temporal modeling.

  • Temporal review-fraud methods detect coordinated anomalies across rating, review-count, and other time series; LocalAR corroborates lead-signal anomalies with supporting signals and succeeded in two Flipkart case studies.CAPT-MDTS detects correlated bursts in average rating, review count, and singleton-review proportion; spatiotemporal features such as travel speed and reviewer-to-venue distance can also signal fraud.Source: §6.1.3 · p. 25

6.2 Detection of fact-based false information

  • Fact-based detection uses engineered features or propagation models: the former draw on content, related information, users, and reshares, while the latter identify anomalous spread patterns.Propagation approaches model how true information spreads, sometimes modeling true and false spread separately or feeding propagation structures into machine-learning models.Source: §6.2 · pp. 25–26

6.2.1 Feature-based detection.

  • Feature-based fact-checking results vary: title text was more informative than article body for distinguishing real from fake news, and satire was harder to distinguish from fake news than real news.Earlier content-based methods performed well, but later studies found content alone less effective, suggesting creators adapt their false information.Source: §6.2.1 · p. 26
  • 0.98 AUC was achieved for Wikipedia hoax detection using all four appearance, network, support, and creator feature categories, versus 0.90 AUC for creator or network features individually [50].The passage attributes the improvement to combining article characteristics with information about its creator and links to existing information.Source: §6.2.1 · p. 27, Figure 14 · p. 27

6.2.2 Detection using propagation modeling.

  • Propagation-based detection models how information spreads across social media, either to characterize the spread of false information or to find ways to prevent it.This subsection introduces these purposes but supplies no specific propagation-detection results in this part.Source: §6.2.2 · p. 27
  • False information dies out when normal users are well connected, but can persist in echo chambers where forceful nodes reinforce previously influenced users.Acemoglu et al.’s model suggests increasing the number and connectivity of normal nodes may reduce propagation.Source: §6.2.2 · p. 28
  • In the SEIZ model, users move among susceptible, exposed, infected, and skeptical states, and a low rate of users leaving exposure predicted rumors in a case study of eight rumors.The model learns parameters from real true- and false-information propagation data; a high exposure-exit rate indicates true information.Source: §6.2.2 · p. 28
  • 78% average accuracy came from adding reply and retweet counts to standard features for Sina Weibo classification, while propagation-tree features helped classify 11,466 rumors.False rumors often began with normal users, gained support from opinion leaders, then spread among normal users; true information was posted by opinion leaders and reposted by normal users.Source: §6.2.2 · p. 28
  • 88% accuracy was achieved for early detection using the first 24 hours of propagation, compared with 72% without propagation information.The propagation-structure classifier achieved 91% accuracy overall.Source: §6.2.2 · pp. 28–29
  • Over 60% lower average rumor lifetime resulted when special nodes spread anti-rumors, and rumor lifetime increased super-linearly with detection delay.Tripathy et al. modeled truth spreading after a delay or through special nodes as soon as they received false information.Source: §6.2.2 · p. 29
  • A 30% increase in true-information delay reduced its reach by 80% of the population, while another mitigation method was 16–41% better than other methods.Nguyen et al. found that few selected spreaders work best when highly influential in large communities; with more selections, influential nodes in smaller communities are more effective.Source: §6.2.2 · p. 29

7 DISCUSSIONS AND OPEN CHALLENGES

The survey covers mechanisms, rationale, impact, characteristics, and detection of fake reviews, hoaxes, and fake news, and identifies open challenges for detecting and mitigating false information.

Dataset availability and comparison

  • Existing algorithms cannot be benchmarked across categories because large-scale public datasets are lacking, and standardized comparisons on Buzzfeed, LIAR [107], CREDBANK, and FakeNewsNet [89] [90] remain undone.Such comparisons are needed to assess algorithms’ advantages and disadvantages and collectively improve the state of the art.Source: §7 · pp. 29–30

Multimedia false information detection

  • Machine learning can generate genuine-looking text, audio, images, and videos, prompting research into signal, vision, and data-analysis methods to detect fabricated or manipulated media.The passage warns that standard detection signals will need reform as generated false information becomes harder for readers to distinguish from true information.Source: §7 · p. 30

Semantic dissonance detection

  • NLP methods could detect semantic dissonance between cited references and claims or between a headline and the main information, which can exploit readers’ aversion to fact-checking.The passage identifies these mismatches as tactics used by deceptive sources to appear credible.Source: §7 · p. 30

Fact-checking from knowledge bases

  • Knowledge-base fact-checking requires maintaining a complete, quality-controlled knowledge base, extracting information from free-form text, and matching it against stored information.Source: §7 · p. 30

Fact-checking using crowdsourcing

  • Crowdsourced reader skepticism and reports can support early fake-news detection [44] and fact-checker resource allocation, since manually checking every story is infeasible and stories can evade filters.Readers may express different emotions when interacting with false versus true information.Source: §7 · p. 30

Bridging echo chambers

  • Data-driven models are needed to bridge conflicting echo chambers and present opposing beliefs effectively, enabling exchange while seeking to reduce polarization.Source: §7 · p. 30

Adversarial creation of false information

  • Because malicious users adapt to detection and prevention countermeasures, research on adaptive adversaries may help mitigate false information’s impact.Source: §7 · p. 30

Mitigation of false information

  • Education about manipulation strategies can improve human detection skills, but effective approaches for “vaccinating” people against false information and scaling them to millions of social-platform users remain open questions.Source: §7 · p. 30

Related papers

Loading 1804.08559v1…